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EXECUTIVE  SUMMARY 

Language  proficiency  standards  and  testing  should  align  with  the  skills  required  for  mission  success. 
Special  Operations  Forces  (SOF)  operators  and  leaders  report  primarily  using  speaking  and  listening 
skills,  as  compared  to  reading  and  writing  skills,  when  deployed  on  missions  [Inside  AOR  Use  of 
Language,  Technical  Report  (#2010011010),  Outside  AOR  Use  of  Language,  Technical  Report 
(#2010011011)].  Furthermore,  the  United  States  Special  Operations  Command  (USSOCOM)  identified 
speaking  as  “the  most  critical  SOF  language  modality”  since  operators  and  leaders  “must  be  able  to 
establish  rapport  and  relations  with  allies  and  partners,  [which  is]  done  primarily  through  conversational 
speaking”  [USSOCOM  M  350-8,  2009,  p.  3],  This  recognition  of  the  importance  of  speaking  proficiency 
for  mission  success  resulted  in  an  USSOCOM  standard  change  from  reading/listening  proficiency 
assessed  with  the  Defense  Language  Proficiency  Test  (DLPT)  to  listening/speaking  proficiency  assessed 
with  the  two-skill  Oral  Proficiency  Interview  (OPI).  Knowledge  of  and  support  for  the  standard  change 
needs  to  be  assessed  to  determine  SOF  operator  and  leader  support  or  opposition.  In  addition,  examining 
reactions  to  the  standard  change  can  identify  gaps  in  communicating  the  change  and  aligning  the  standard 
with  training  and  incentive  programs. 

Since  language  proficiency  testing  should  align  with  the  modality  used  on  missions,  the  USSOCOM 
identified  the  OPI,  a  measure  of  speaking  and  participatory  listening  proficiency,  as  the  preferred  test  for 
assessing  an  operator’s  language  proficiency  (USSOCOM  M  350-8,  2009).  This  report  evaluates  SOF 
operator  and  leader  reactions  to  the  standard  change  and  OPI  so  that  SOF  leadership,  CLPMs,  unit 
leaders,  testing  center  administrators,  and  test  developers  can  use  the  SOF  operator  and  leader  feedback  to 
improve  the  OPI,  OPI  testing  conditions,  and  the  match  between  SOF  operator  language  testing,  language 
training,  and  incentive  programs. 

Reactions  to  the  Standard  Change 

The  standard  change  has  received  support  from  many  SOF  operators  (45%,  n  =  468)  and  leaders  (54%,  n 
=  434).  However,  some  SOF  operators  and  leaders  reported  neutral  [SOF  operators  (45%,  n  =  464),  SOF 
leaders  (37%,  n  =  296)]  and  opposing  [SOF  operators  (10%,  n  =  107),  SOF  leaders  (9%,  n  =  70)] 
reactions  to  the  standard  change.  SOF  operator  and  leader  comments  suggested  three  reasons  for  these 
neutral  and  opposing  opinions: 

•  First,  many  respondents  (Supported:  n  =  50,  Neutral:  n  =  14,  Opposed:  n  =  35)  commented  that 
reading  is  still  an  important  skill  for  mission  success  and  should  be  included  in  language  testing. 
This  suggests  that  a  reading  test  may  be  necessary  for  some  SOF  operators. 

•  Second,  some  were  concerned  that  SOF  operator  language  training  (Supported:  n  =  17,  Neutral:  n 
=  5,  Opposed:  n  =  10)  and  language  proficiency  incentive  programs  (Supported:  n  =  17,  Neutral: 
n  =  13,  Opposed:  n  =  10)  had  not  been  updated  to  align  with  OPI  testing,  so  SOF  operators  would 
not  be  prepared  for  OPI  testing  nor  would  they  be  motivated  to  take  it.  Thus,  ensuring  SOF 
operators  are  prepared  for  the  OPI  and  paid  for  OPI  scores  at  the  appropriate  levels  should  be 
addressed  within  all  components.  Communicating  how  the  OPI  is  used  to  pay  Foreign  Language 
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Proficiency  Bonus  -  currently  and  as  policy  changes  -  will  be  key  to  addressing  this  concern  for 
SOF  personnel. 

•  Third,  unfamiliarity  with  the  change  (Supported:  n  =  20,  Neutral:  n  =  35,  Opposed:  n  =  1)  and/or 
inexperience  with  OPI  testing,  particularly  two-skill  OPI  testing,  (Supported:  n  =  19,  Neutral:  n  = 
19,  Opposed:  n  =  4)  were  mentioned  as  reasons  why  SOF  operators  and  leaders  were  undecided 
about  the  standard  change.  This  suggests  that  information  about  the  standard  change  and 
associated  testing  should  be  widely  disseminated. 

OPI  Testing  Background 

Overall,  44%  ( n  =  505)  of  SOF  operators  reported  having  taken  an  official  OPI,  and  77%  (n  =  448)  of 
these  SOF  operators  reported  not  encountering  any  problems  during  testing.  For  those  who  experienced 
problems,  the  most  commonly  reported  problems  were  disruptions  while  testing,  delays/problems 
receiving  score  feedback,  and  technical  difficulties,  such  as  problems  hearing  the  tester  on  the  phone. 
Disruptions  may  suggest  a  lack  of  sufficient  control  and/or  standardization  in  the  testing  environments  or 
that  facilities  are  not  structured  for  OPI  testing.  This  should  be  investigated.  These  findings  suggest  the 
OPI  often  functions  properly,  but  the  occasional  problems  that  do  arise  should  be  addressed  as  they  are 
likely  to  become  more  frequent  as  OPI  testing  increases  with  the  standard  change  and  strains  on  OPI 
testing  resources. 

OPI  Perceptions 

For  the  SOF  operators  who  have  taken  an  OPI  (44%,  n  =  505)  and  the  SOF  leaders  who  were  familiar 
with  it  and  could  comment  on  their  operators’  ratings  (12%,  n  =  88),  most  had  neutral  or  favorable 
perceptions  of  the  OPI  in  terms  of  its  job-relatedness  and  accuracy.  Moreover,  SOF  operators  and  leaders 
who  supported  the  standard  change  agreed  more  strongly  that  the  OPI  accurately  assesses  and  relates  to 
what  SOF  operators  do  on  missions.  However,  many  SOF  operators  and  leaders  agreed  that  there  are 
language  tasks  SOF  operators  need  to  perform  not  covered  by  the  OPI,  suggesting  it  needs  to  cover  more 
mission-specific  tasks.  Although  many  SOF  operators  and  leaders  believe  the  OPI  relates  to  what  SOF 
operators  do  on  the  job  and  accurately  assesses  their  ability  to  use  the  language  while  deployed,  some  also 
believe  it  does  not  cover  all  the  skills  SOF  operators  need  for  mission  success.  Since  the  OPI  is  a 
measure  of  functional  speaking  proficiency  (and  participatory  listening  proficiency  in  the  case  of  the  two- 
skill  OPI),  more  mission-related  prompts  and  role  plays  could  be  introduced  into  the  OPI  without 
compromising  the  assessment  of  proficiency  tasks.  However,  this  would  require  a  modification  of  the 
assessment  and  retraining  of  the  OPI  testers. 

In  addition  to  their  favorable  OPI  perceptions,  68%  of  SOF  leaders  reported  they  pay  attention  to  SOF 
operator  OPI  ratings,  56%  indicated  that  SOF  operator  OPI  ratings  are  important  or  very  important,  and 
45%  often  or  very  often  encourage  SOF  operators  to  study  and  do  well  on  the  OPI.  It  appears  that  while 
some  SOF  leaders  have  shifted  their  focus  to  OPI  testing,  many  are  still  transitioning  to  the  focus  on 
speaking  proficiency  and  OPI  testing.  This  may  be  a  result  of  unfamiliarity  with  the  standard  change  on 
the  part  of  SOF  leaders.  However,  these  results  should  be  inteipreted  with  caution  because  of  the  small 
number  of  SOF  leaders  who  responded  to  these  items.  As  OPI  testing  continues  to  increase,  the  amount 
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of  attention  and  importance  SOF  leaders  place  on  OPI  testing  and  scores  should  be  monitored  to  see  if 
they  also  increase. 

See  Appendix  A  for  details  the  2009  SOF LCN A  Project.  For  questions  or  more  information  about  the 
SOFLO  and  this  project,  please  contact  Mr.  Jack  Donnelly  (iohn.donnelly@socom.mil).  For  specific 
questions  related  to  data  collection  or  reports  associated  with  this  project,  please  contact  Dr.  Eric  A. 
Surface  (esurface@swa-consulting.com)  or  Dr.  Reanna  Poncheri  Harman  (rpharman@swa- 
consulting.com)  with  SWA  Consulting  Inc. 
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SECTION  I:  REPORT  AND  PROJECT  OVERVIEW 
Oral  Proficiency  Interview  (OPI)  Report  Purpose 

The  United  States  Special  Operations  Command  (USSOCOM)  identified  speaking  proficiency  as  the 
language  skill  most  important  to  mission  success  (USSOCOM  M  350-8,  2009).  Input  from  Special 
Operations  Forces  (SOF)  operators  and  leaders  in  the  field  confirm  the  importance  of  speaking 
proficiency  for  mission  success  and  job  performance  [{Inside  AOR  Use  of  Language,  Technical  Report 
(#2010011010),  ( Outside  AOR  Use  of  Language,  Technical  Report  (#201001101 1)].  To  align  training 
and  testing  with  the  mission-related  proficiency  requirements,  USSOCOM  changed  the  testing  standard 
from  reading/listening  assessed  with  the  Defense  Language  Proficiency  Test  (DLPT)  to 
listening/speaking  proficiency  assessed  with  the  two-skill  Oral  Proficiency  Interview  (OPI).  The  purpose 
of  this  report  is  to  assess  support  for  the  standard  change  and  perceptions  of  the  new  test  of  record,  the 
OPI. 

The  standard  change  occurred  in  2009.  The  SOF  community’s  support  for  the  standard  change  should  be 
assessed  to  identify  the  advantages  and  disadvantages  associated  with  a  focus  on  speaking  proficiency,  as 
well  as  the  acceptance  of  the  change.  This  information  can  also  detect  areas  where  the  standard  change 
has  not  been  applied,  such  as  where  language  training  or  incentive  programs  have  not  been  updated. 

As  the  SOF  community  transitions  toward  the  OPI,  their  experiences  with  and  perceptions  of  the  test  need 
to  be  examined  since  identifying  any  potential  issues  now  allows  them  to  be  addressed  prior  to  more 
widespread  OPI  testing.  As  demand  for  OPI  testing  increases  as  a  result  of  the  standard  change,  logistical 
and  other  problems  with  the  OPI  testing  process  -  if  they  exist  -  will  only  be  exacerbated.  First, 
examining  experiences  of  SOF  personnel  identifies  if  problems  occur  with  the  OPI  delivery,  as  negative 
testing  experiences  could  hinder  USSOCOM’s  efforts  to  promote  speaking  proficiency.  SOF  operators 
and  leaders  who  have  negative  testing  experiences  may  be  less  likely  to  take  the  test  seriously  than  those 
who  have  positive  testing  experiences.  Second,  if  problems  are  occurring,  the  nature  of  the  issues  must  be 
identified  so  corrective  action  can  be  taken.  Third,  perceptions  of  an  assessment’s  accuracy  and  job¬ 
relatedness  can  influence  motivation  and  attitudes  toward  the  test  (Chan,  Schmitt,  Jennings,  Clause,  & 
Delbridge,  1998;  Gilliand  1993;  Smither,  Millsap,  Stoffey,  Reilly,  &  Pearlman,  1996).  SOF  operator  and 
leader  perceptions  of  the  OPI  could  impact  test  performance,  with  those  who  believe  the  OPI  is  inaccurate 
and  not  job-related  being  less  motivated  to  do  well  (or  to  encourage  others  to  do  well)  on  the  test.  Finally, 
by  asking  test  users,  factors  and  issues  not  anticipated  by  test  developers  and  policymakers  are  often 
discovered.  Collecting  this  type  of  user  feedback  should  be  part  of  a  test  development  and  maintenance 
plan. 

This  report  presents  SOF  operator  and  leader  perspectives  on  the  standard  change  and  SOF  operator  OPI 
testing.  Section  I  provides  an  overview  of  this  report  and  the  SOF  Language  and  Culture  Needs 
Assessment  (LCNA)  Project.  Section  II  provides  reactions  to  the  standard  change.  Section  III  describes 
SOF  operator  OPI  testing  backgrounds.  Sections  IV  reports  SOF  operator  and  leader  OPI  testing 
perceptions.  Section  V  provides  conclusions  based  on  Sections  II  through  IV.  Appendix  A  details  the 
2009  SOF  LCNA  Project,  and  Appendix  B  discusses  the  report  methodology,  including  participants, 
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measures,  and  analyses.  Appendices  C  through  E  provide  additional  results  by  component.  Appendix  F 
provides  MI  linguists  and  09L  responses  and  Appendix  G  provides  responses  for  respondents  currently  in 
the  training  pipeline.  Appendices  H  and  I  present  additional  OPI  perception  comparisons  for  SOF 
operators  and  leaders.  Appendix  J  defines  all  comment  codes  used  in  the  report. 

LCNA  Project  Purpose 

The  Special  Operations  Forces  Language  Office  (SOFLO)  commissioned  the  2009  SOF  LCNA  Project  to 
gain  insights  on  language  and  culture  capability  and  issues  across  the  United  States  Special  Operations 
Command  (USSOCOM).  The  goal  of  this  organizational-level  needs  assessment  is  to  inform  strategy  and 
policy  to  ensure  SOF  personnel  have  the  language  and  culture  skills  needed  to  conduct  their  missions 
effectively.  Data  were  collected  between  March  and  November  2009  from  personnel  in  the  SOF 
community,  including  operators  and  leaders.  Findings,  gathered  via  focus  groups  and  a  web-based 
survey,  will  be  presented  in  a  series  of  reports  divided  into  three  tiers.  The  specific  reports  in  each  of 
these  tiers  will  be  determined  and  contracted  by  the  SOFLO.  Tier  I  reports  focus  on  specific,  limited 
issues  (e.g.,  Inside  AOR  Use  of  Language).  Tier  II  reports  integrate  and  present  the  most  important 
findings  across  related  Tier  I  reports  (e.g.,  Use  of  Language  and  Culture  on  Deployment )  while  including 
additional  data  and  analysis  on  the  topic.  One  Tier  III  report  presents  the  most  important  findings, 
implications,  and  recommendations  across  all  topics  explored  in  this  project.  The  remaining  Tier  III 
reports  present  findings  for  specific  SOF  organizations  [e.g.,  Air  Force  Special  Operations  Command 
(AFSOC),  Special  Forces  (SF)  Command].  Two  foundational  reports  document  the  methodology  and 
participants  associated  with  this  project.  Report  topics  are  determined  by  the  SOFLO  and  are  subject  to 
change. 

Relationship  of  Oral  Proficiency  Interview  (OPI)  to  the  LCNA  Project 

Oral  Proficiency  Interview  (OPI)  is  a  Tier  I  report  that  will  be  integrated  with  two  other  Tier  /  reports, 
Defense  Language  Proficiency  Test  ( DLPI )  and  DLAB  (Defense  Language  Aptitude  Battery): 
Perspectives  from  the  Field,  into  a  Tier  II  report,  Testing/Metrics  (see  Appendix  A  for  the  initially 
planned  report  structure).  The  final  reports  produced  will  be  determined  by  the  SOFLO. 
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SECTION  II:  REACTIONS  TO  THE  STANDARD  CHANGE 

In  2009,  USSOCOM  announced  a  standard  change  from  reading/listening  assessed  with  the  Defense 
Language  Proficiency  Test  (DLPT)  to  listening/speaking  assessed  with  the  two-skill  Oral  Proficiency 
Interview  (OPI).  It  is  important  to  examine  SOF  operator  and  leader1  reactions  to  this  change  to 
determine  how  many  support  or  oppose  it.  This  section  describes  the  degree  to  which  SOF  operators  and 
leaders  support  the  standard  change,  and  any  specific  feedback  they  have  related  to  it. 

Research  Questions 

This  section  addresses  the  following  questions: 

•  How  did  SOF  operators  and  leaders  react  to  the  standard  change? 

•  How  do  SOF  operator  and  leader  reactions  compare? 

Main  Findings 

Overall,  many  SOF  operators  (45%,  n  =  468)  and  leaders  (54%,  n  =  434)  supported  the  standard  change 
(SOF  leaders  reported  significantly  greater  support  than  SOF  operators)  and  very  few  opposed  it. 
Respondents  supporting  the  standard  change  indicated  that  speaking  and  listening  are  the  most  important 
modalities  for  SOF  operators,  and  they  supported  the  focus  on  skills  used  on  deployment.  However, 
many  operators  (45%,  n  =  464)  and  leaders  (37%,  n  =  296)  were  neutral  about  their  reaction  to  the 
standard  change.  Feedback  suggests  that  this  neutrality  stems  from:  (1)  unawareness  of  the  standard 
change  (n  =  35),  (2)  unfamiliarity  with  the  two-skill  OPI  {n  =  19),  and  (3)  belief  that  reading  proficiency 
should  also  be  tested  (n  =14).  In  addition  to  also  believing  that  reading  proficiency  should  be  tested  («  = 
35),  respondents  who  opposed  the  standard  change  also  wanted  a  financial  incentive  for  OPI  testing  ( n  = 
10)  and  training  to  match  the  new  testing  format  ( n  =  10).  Given  that  the  standard  changed  in  the  same 
year  the  survey  was  administered,  support  from  both  SOF  operators  and  leaders  is  encouraging.  Even  the 
large  number  of  SOF  operators  and  leaders  undecided  about  the  standard  change  is  to  be  expected  since  it 
is  still  relatively  new  information.  There  were  no  significant  differences  found  across  SOF  components. 2 
Results  suggest  that  more  information  dissemination  of  the  standard  change  and  associated  testing  policy 
is  needed. 

Detailed  Findings 

Overall,  findings  are  positive  since  there  is  little  opposition  from  both  SOF  operators  (10%,  n  =  107)  and 
leaders  (9%,  n  =  70)  to  the  USSOCOM’s  standard  change.  Most  SOF  operators  and  leaders  reported 
neutrality  (45%,  n  =  464  and  37%,  n  =  296,  respectively)  or  support  (45%,  n  =  468  and  54%,  n  =  434, 
respectively)  for  the  change.  More  SOF  operators  than  SOF  leaders  were  neutral,  while  more  SOF 


1  When  referring  to  SOF  operators  and  leaders,  this  section  focuses  only  on  the  SOF  operators  and  leaders  who 
participated  in  the  survey.  Please  see  Appendix  B  (Methodology)  and  the  Participation  Report  (Technical  Report 
#201001 1003)  for  more  information  about  survey  respondents. 

2  Appendix  C  (p.  33)  provides  comparisons  between  components  for  the  standard  change  items  for  SOF  operators 
and  leaders. 
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leaders  than  SOF  operators  were  supportive  (Figure  1,  p.  9).  Comparing  the  two  groups,  SOF  leaders 
reported  significantly  greater  support  for  the  standard  change  than  SOF  operators.3 


Figure  1.  SOF  Operator  and  Leader  Reactions  to  the  Standard  Change 


50% 


Strongly  Somewhat  Neither  Somewhat  Strongly 

Oppose  Oppose  Oppose  nor  Support  Support 

Support 


■  SOF  Leader 

■  SOF  Operator 


Comments  from  those  who  supported  the  standard  change  (Table  1,  p.  1 1)  along  with  comments  from 
focus  group  participants  (Table  2,  p.  1 1)  indicated  that  speaking  and  listening  are  the  most  important 
modalities  that  operators  use  while  deployed. 


“I feel  that  listening  and  speaking  are  more  important  to  the  operator  while  on 
deployment  than  listening  and  reading” 

SOF  Operator,  Other  Assignment 
Supported  the  standard  change 

“It  is  practical  to  change  the  format  to  a  listen  and  speaking.  This  is  what  the  bulk  of  the 
operators  will  engage  in  while  on  their  deployments.  ” 

SOF  Leader,  7th  Special  Forces  Group  (SFG) 
Supported  the  standard  change 


“we  listen  and  speak  on  deployments  more  than  read ’ 

SOF  Operator,  95th  Civil  Affairs  Brigade  (CAB) 
Supported  the  standard  change 

“The  purpose  of  language  for  all  soldiers  is  to  make  them  capable  of  engaging  locals  to 
effectively  accomplish  the  mission.  Nothing  could  be  more  important  in  regards  to 
language  than  listening  and  speaking  skills.  ” 

SOF  Leader,  5th  SFG 
Supported  the  standard  change 


3 1(1,744.69)  =  -3.07,/?  <.001 
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“  ...with  the  mention  of  everything  going  to  the  OPI,  that ’s  the  first  I’ve  heard  of  it.  I 
think  that ’s  a  great  idea  because,  as  you  said,  we  speak  more  than  we  read.  And  I  think 
that ’s  a  really  great  idea.  ” 

SOF  Operator,  10th  SFG  focus  group 

Moderator:  “You  guys  did  relatively  well  in  speaking,  do  you  feel  like  that ’s  your  best  skill?  The 
one  that  you  developed  the  most?” 

Participant:  “At  that  point  I  don  ’t  know  if  it  was  necessarily  better  than  my  reading, 
that ’s  just  my  way  of  learning,  but  I  think  it ’s  definitely  the  most  useful,  I  think  it  should 
be  the  standard — I  think  it  should  be  the  most  important  part.  ” 

SOF  Operator,  1st  SFG  focus  group 
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Table  1.  SOF  Operator  and  Leader  Reactions  to  the  Standard  Change4 


Oppose 

Neutral 

Support 

Total 

Total 

Operators 

Leaders 

Operators 

Leaders 

Operators 

Leaders 

Operators 

Leaders 

Supporting  change  -  General 

1 

2 

5 

6 

56 

38 

62 

46 

108 

Speaking  and  listening  are  the  most 
important  proficiencies  for  operators 

2 

1 

3 

4 

40 

49 

45 

54 

99 

Need  listening,  speaking,  and  reading 

13 

22 

7 

7 

24 

26 

44 

55 

99 

Not  familiar  with  change 

0 

1 

13 

22 

9 

11 

22 

34 

56 

None/NA/off  topic 

6 

2 

21 

9 

6 

5 

33 

16 

49 

Opposing  change  -  General 

12 

14 

4 

4 

6 

6 

22 

24 

46 

Not  familiar  with  the  OPI 

0 

4 

9 

10 

8 

11 

17 

25 

42 

Not  getting  paid  for  OPI  or  want  a 
financial  incentive 

3 

7 

3 

10 

5 

12 

11 

29 

40 

Skills  should  be  relevant  to  their  job 

0 

1 

3 

2 

14 

14 

17 

17 

34 

Need  to  match  training  to  testing  format 

7 

3 

5 

3 

6 

11 

18 

17 

35 

Against  reading/DLPT 

1 

1 

0 

2 

6 

7 

7 

10 

17 

Total 

45 

58 

73 

79 

180 

190 

298 

327 

625 

Table  2.  Focus  Group  Reactions  to  the  Standard  Change 


Comments 

The  test  should  focus  on  speaking 

50 

The  test  should  focus  on  listening 

29 

The  test  should  focus  on  reading 

14 

Lack  experience  with  the  OPI 

9 

Confusion  between  OPI  and  OPE 

4 

The  test  should  focus  on  conversational  listening 

3 

Positive  implications  of  standard  change 

2 

Negative  implications  of  standard  change 

1 

The  test  should  focus  on  passive  listening 

1 

4  Totals  may  differ  from  others  throughout  this  section  since  Table  1  only  includes  SOF  operators  and  leaders  who  also  commented  on  why  they  supported, 
opposed,  or  were  neutral  regarding  the  standard  change. 
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SOF  leaders  and  operators  who  were  either  neutral  or  opposed  the  standard  change  reported  multiple 
reasons  for  their  lack  of  support.  First,  some  reported  that  reading  proficiency  should  still  be  tested, 
which  was  also  mentioned  by  participants  who  supported  the  change. 

“A  good  DLPT would  include  speaking,  listening  and  reading” 

SOF  Operator,  AFSOC 
Neutral  on  the  standard  change 

‘‘Listening  and  reading  are  still  important  skills  to  maintain.  There  should  be  a  balance 
developed  to  test  on  all  4  skills.  ” 

SOF  Leader,  AFSOC 
Opposed  the  standard  change 

“I  still  think  that  all  three  elements  should  be  tested.  You  can 't  take  away  one  and  expect 
the  results  of  a  mission  to  change.  The  soldier  would  still  be  lacking  in  one  area.  ” 

SOF  Operator,  4th  Psychological  Operations  Group  (POG) 

Supported  the  standard  change 

Second,  others  reported  that  training  should  change  to  match  the  new  standard  and  emphasize  speaking 
proficiency  and  the  OPI. 

‘‘You  had  better  do  a  lot  more  speaking  training,  since  most  of  SWC  is  about  reading.  ” 

SOF  Operator,  Other  Assignment 
Neutral  on  the  standard  change 

“Speaking  the  language  is  the  most  difficult  task,  if  we  are  going  to  require  a  speaking 
test,  the  our  training  should  involve  immersion  in  the  target  language.  Otherwise  we  are 
going  to  set  our  Soldiers  up  for  failure.  ” 

SOF  Leader,  4th  POG 
Opposed  the  standard  change 

Third,  some  participants  also  commented  that  incentives  should  be  offered  for  testing  on  the  OPI. 

“Unfortunately,  the  Two-Skill  OPI  does  not  qualify  you  for  Language  Proficiency  Pay.  If 
it  did,  then  there  would  be  more  support  for  it.  Therefore,  team  members  will  give  it 
much  less  consideration  than  the  DLPT.  Some  members  do  not  try  to  pass  the  DPLT 
because  they  know  they  have  not  had  the  time  to  study  enough  to  receive  a  high  enough 
score  to  warrant  Language  Pay.  Therefore,  they  are  content  to  fail  it  and  use  their  time 
on  other  maintaining  other  skill  sets.  ” 

SOF  Leader,  USASOC  HQ 
Neutral  on  the  standard  change 
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“It  is  ok  as  long  as  SOF  personnel  can  still  get  paid  for  it.  Why  take  an  OPI,  when  the 
soldier  has  to  take  the  DLPT  as  well.  Seems  like  a  waste  of  time,  twice  as  much  testing. 

It  seems  as  though  the  SOF  soldier  is  being  constrained  when  it  comes  to  being  afforded 
the  chance  to  earn  language  pay.  ” 

SOF  Operator,  1st  SFG 
Opposed  the  standard  change 

Communication  of  the  standard  change  is  important  to  USSOCOM’s  policy  of  encouraging  the 
development  of  speaking  proficiency.  Some  SOF  operators  and  leaders,  particularly  those  neutral 
regarding  the  standard  change,  indicated  that  they  were  unfamiliar  with  the  standard  change  and/or  the 
OPI.  This  suggests  more  information  dissemination  is  needed. 

“I  did  not  know  that  the  standard  had  changed,  which  illustrates  some  of  the  problems 
with  the  overall  SOF  language  program  ’’ 

SOF  Leader,  7th  SFG 
Neutral  on  the  standard  change 

“I  have  had  no  experience  with  this  test,  yet  it  seems  like  it  would  be  more  relevant  to  the 
job  we  conduct  as  SF  Operators.  ” 

SOF  Operator,  Other  Assignment 
Neutral  on  the  standard  change 
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SECTION  III:  SOF  OPERATOR  OPI  TESTING  BACKGROUND 

Part  of  the  standard  change  included  recognition  of  the  Oral  Proficiency  Interview  (OPI),  particularly  the 
two-skill  OPI  (speaking  and  participatory  listening),  as  the  test  of  record  for  assessing  language 
proficiency  (USSOCOM  M  350-8,  2009).  It  is  important  to  examine  how  many  SOF  operators5  have 
experience  with  OPI  testing.  In  addition,  any  problems  encountered  during  OPI  testing  need  to  be 
identified  so  they  can  be  addressed  and  potentially  eliminated  since  existing  problems  will  likely  only  be 
exacerbated  as  demand  for  OPI  testing  increases.  This  section  presents  SOF  operators’  OPI  testing 
experiences. 

Research  Questions 

This  section  addresses  the  following  questions: 

•  How  many  SOF  operators  have  taken  an  official  OPI? 

•  Have  these  SOF  operators  experienced  problems  taking  the  OPI? 

o  If  so,  what  types  of  problems  have  they  experienced? 

Main  Findings 

Nearly  half  of  SOF  operators  reported  having  taken  an  official  OPI  (44%,  n  =  505),  and  most  of  these 
SOF  operators  have  not  encountered  problems  during  testing  (77%,  n  =  448).  Of  those  who  experienced 
problems,  commonly  reported  issues  included  disruptions  while  testing,  delays/problems  receiving 
feedback  on  their  score,  and  technical  problems  (e.g.,  not  being  able  to  hear  the  tester  on  the  phone). 
Operators  who  last  took  an  OPI  in  2009  reported  experiencing  more  technical  problems,  delays/problems 
receiving  feedback,  and  other  problems  than  operators  who  last  took  an  OPI  prior  to  2009.  While  this 
could  be  due  to  the  standard  change  and  an  increase  in  demand  for  the  OPI,  these  issues  should  be 
investigated  for  correction  since  more  OPI  testing  will  be  occurring. 

Detailed  Findings 

SOF  Operator  OPI  Testing  Background 

Overall,  44%  (n  =  505)  of  SOF  operators  report  they  have  taken  an  official  OPI  (Figure  2,  p.  15),  with 
most  reporting  they  last  took  an  OPI  in  2008  (n  =  1 10)  and  2009  (n  =  95;  Figure  3,  p.  15).  Many  SOF 
operators  (n  =  186)  report  taking  an  OPI  prior  to  2006. 6  Most  historic  OPI  testing  likely  took  place  at  the 
end  of  initial  acquisition  training  (IAT). 


5  When  referring  to  SOF  operators,  this  section  focuses  only  on  the  SOF  operators  who  participated  in  the  survey 
and  reported  they  had  taken  an  official  OPI.  Please  see  Appendix  B  (Methodology)  and  the  Participation  Report 
(Technical  Report  #2010011003)  for  more  information  about  survey  respondents. 

6  Since  the  data  prior  to  2006  could  not  be  broken  down  by  year,  it  is  mentioned  here  solely  as  a  point  of 
information. 
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Figure  2.  SOF  Operators’  OPI  Experience 


2006  2007  2008  2009 


SOF  Operator  OPI  Testing  Experiences 

Overall,  77%  (n  =  448)  of  SOF  operators  experienced  no  problems  during  OPI  testing.  Of  those  few  SOF 
operators  that  experienced  testing  problems,  the  most  commonly  reported  issue  was  disruptions  during 
testing.  Disruptions  may  suggest  a  lack  of  sufficient  control  and/or  standardization  in  the  testing 
environments  or  that  facilities  are  not  structured  for  OPI  testing,  which  should  be  investigated.  Other 


7  Only  includes  SOF  operators  who  reported  last  taking  an  OPI  since  2006.  Data  prior  to  2006  were  not  included 
because  the  data  could  not  be  separated  by  year. 
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problems  SOF  operators  occasionally  encountered  included  delays/problems  receiving  feedback  on  their 
score  and  technical  problems  (Figure  4,  p.  16).  Non-frequent  issues  included  test-scheduling  delays, 
problems  accessing  the  test  center,  and  “other”  issues.  Many  SOF  operators  who  experienced  “other” 
problems  during  testing  reported  technical  difficulties  and  problems  interacting  with  the  OPI  interviewer. 


Figure  4.  OPI  Testing  Experiences. 


Area 

Reported  Testing  Problems 

Percentage  of  Total  Responses 

1 

None 

77% 

2 

Disruptions  while  taking  test 

5% 

3 

Delays/problems  receiving  feedback  on  your  score 

3% 

4 

Technical  issues 

3% 

5 

Test  scheduling  delays 

3% 

6 

Problems  accessing  testing  centers 

1% 

7 

Other 

3% 

8 

Technical  difficulties 

1% 

9 

Other 

1% 

10 

Tester  problems 

1% 

11 

Different  dialect  used  in  the  field 

1% 

12 

Problems  with  scoring 

1% 

13 

Structure  of  the  test 

1% 

Note.  Testing  problems  presented  in  italics  were  coded  from  responses  to  the  “other”  response  category.  Percentages  may  not  sum  to  100%  as 
respondents  could  select  more  than  one  problem. 


Respondents’  follow-up  comments  elaborate  on  the  problems  they  experienced  during  OPI  testing  (Table 
3,  p.  18).  For  example,  some  of  the  disruptions  during  testing  included  problems  in  the  testing 
environment: 

“The  testing  center  had  background  noise  and  disruptions  from  the  building  and  the 
vehicle  traffic  outside  the  building’’ 
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SOF  Operator,  7th  SFG 

Some  of  the  participants  commented  on  problems  with  scoring,  often  suggesting  they  did  not  believe  it 
was  a  fair  system: 


“GREAT  TEST  BUT  HORRIBLE  GRADING  SCALE.  EVERY  PERSON  IN  OUR 
LANGUAGE  RECEIVED  THE  SAME  SCORE  EVEN  THOUGH  MEMBERS  DLPT 
SCORES  RANGED  FROM  0+  TO  3  ” 

SOF  Operator,  1st  SFG 


Many  technical  problems  reported  appear  to  be  telephone  audio  problems: 

“The  telephone  connection  made  it  nearly  impossible  to  understand  the  individual  on  the 
other  end  of  the  line.  I  continually  had  to  ask  the  tester  to  repeat  themselves  due  to  the 
poor  connection  which  was  interpreted  as  me  not  understanding  the  original  question  or 
comment  made  by  the  instructor’' 

SOF  Operator,  Deployed  SO  Unit 

Focus  group  participants  also  discussed  some  issues  during  OPI  testing,  including  scheduling  difficulties: 

“The  testing  thing,  I  can  tell  you  why  that  happened.  You  only  schedule  them  on 
Monday,  and  they  only  have  24  slots  every >  Monday.  So  that ’s  why  you  get  two  battalions 
trying  to  schedule  guys  all  the  time.  I’m  trying  to  schedule  our  company  right  now,  and  I 
can ’t  do  it  until  June.  And  that ’s  one  of  the  biggest  problems.  ’’ 

SOF  Operator,  1st  SFG 

Problems  interacting  with  the  interviewer  also  came  up  during  the  focus  groups: 

Participant:  “[...]  you  have  to  know  the  language,  but  it’s  also  how  you  build  rapport 
with  the...  ” 

Moderator:  “With  the  test  administrator?  ” 

Participant:  “Right,  right.  Because  I  know  we  had  a  Marine  who  was  testing  on  Iraqi  and 
there  was  like  Shias  and  all  that  stuff.  They  basically  clashed  over  the  phone.  He  got  a 
really  good  score  on  the  DLPT,  he  had  a  2/3  on  the  DLPT,  but  on  the  OPI  he  just,  there 
was  nothing.  ’’ 

Moderator:  “There  was  some  tension  there.  ” 

Participant:  “Right,  right,  I  think  he  got  an  0/1  or  something.  ’’ 

SOF  Operator, 

Marine  Corps  Forces  Special  Operations  Command  (MARSOC)  focus  group 
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Table  3.  Elaboration  of  OPI  Testing  Problems 


Percentage  of  Total  Responses 

Process  problems 

27% 

NA/None/Not  relevant 

19% 

Telephone/audio  problems 

19% 

Problems  with  scoring 

15% 

Speaking  and  listening  are  important/Should  be  part  of  the  DLPT 

15% 

Should  be  done  face-to-face 

8% 

Total  N 

26 

Note.  Percentages  may  not  sum  to  100%  as  respondents  could  comment  on  more  than  one  topic. 


Comparing  SOF  operators  who  reported  their  last  OPI  was  between  2006  and  2009,  more  SOF  operators 
who  last  took  an  OPI  in  2009  (31%)  encountered  testing  problems  than  SOF  operators  who  last  tested  in 
2006  (14%),  2007  (11%),  and  2008  (17%;  Table  4,  p.  18).  It  appears  the  increase  in  testing  problems 
resulted  from  increases  in  technical  issues  and  delays/problems  receiving  score  feedback.  The  higher 
percentage  of  issues  reported  by  SOF  operators  who  last  took  an  OPI  in  2009  may  be  related  to  the 
standard  change  and  increase  in  demand  for  the  OPI.  However,  it  may  be  attributable  to  other  causes, 
such  as  lack  of  representativeness  in  the  sample  of  survey  participants  who  last  took  the  OPI  in  each  of 
these  years.  Another  possibility  is  that  those  who  recently  took  the  OPI  remember  problems  encountered 
better  than  those  who  took  the  test  previously. 


Table  4.  Time  of  Last  OPI  by  Testing  Problems8 


2006 

2007 

2008 

2009 

None 

86% 

89% 

83% 

69% 

Other 

2% 

5% 

5% 

12% 

Delays/problems  receiving  feedback  on  your  score 

0% 

0% 

6% 

11% 

Technical  issues 

3% 

2% 

1% 

11% 

Disruptions  while  testing 

5% 

5% 

8% 

8% 

Test  scheduling  delays 

0% 

2% 

4% 

4% 

Problems  accessing  testing  centers 

5% 

2% 

0% 

2% 

Total  N 

44 

57 

109 

95 

Note.  Percentages  may  not  sum  to  1 00%  as  respondents  could  select  more  than  one  problem. 


8  Only  includes  SOF  operators  who  reported  last  taking  an  OPI  since  2006. 
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SECTION  IV:  OPI  TESTING  PERCEPTIONS 

How  much  SOF  leaders  pay  attention  to  SOF  operator  language  testing  and  view  it  as  important  can 
impact  SOF  operators’  motivation  to  perform  well  on  the  test  as  well  as  SOF  operator  perceptions  of 
language  testing.  Since  USSOCOM  implemented  a  standard  change  to  assess  language  proficiency 
through  Oral  Proficiency  Interview  (OPI)  testing,  the  number  of  SOF  leaders  who  have  shifted  their  focus 
to  SOF  operator  OPI  testing  should  be  explored  as  an  indication  of  the  SOFLO  community’s  application 
of  USSOCOM’s  standard  change.  One  purpose  of  this  section  is  to  document  how  SOF  leaders  perceive 
their  operators’  OPI  ratings. 

It  is  also  important  to  examine  SOF  operator  and  leader9  perceptions  of  the  Oral  Proficiency  Interview 
(OPI).  Research  has  found  that  test  takers  who  perceive  tests  as  more  accurate  have  higher  motivation  to 
perform  well  (Hausknechtet,  Day,  &  Thomas,  2004).  Additionally,  test  takers  who  perceive  tests  as  more 
related  to  their  jobs  also  have  higher  motivation  to  perform  well.  Test  taker  motivation  is  important  since 
motivated  test  takers  are  more  likely  to  engage  and  demonstrate  their  full  capability  and,  therefore,  the 
test  will  function  as  designed,  providing  an  accurate  assessment  of  speaking  proficiency  in  this  case.  A 
second  purpose  of  this  section  is  to  examine  SOF  operator  and  leader  accuracy  and  job-re latedness 
perceptions  of  the  Oral  Proficiency  Interview  (OPI)  to  determine  if  these  are  areas  of  concern  for  future 
testing. 

Research  Questions 

This  section  addresses  the  following  questions: 

•  How  many  SOF  leaders  are  familiar  with  the  OPI? 

•  How  do  SOF  leaders  perceive  their  operators’  OPI  ratings? 

o  Do  leaders  pay  attention  to  their  operators’  OPI  ratings? 
o  How  important  are  operator  OPI  ratings  to  leaders? 

o  How  often  do  leaders  encourage  operators  to  study  and  do  well  on  the  OPI? 

•  Do  SOF  operators  and  leaders  perceive  the  OPI  as  a  job-related  and  accurate  assessment  of 
speaking  proficiency? 

Main  Findings 

For  SOF  leaders,  59%  ( n  =  442)  reported  familiarity  with  the  OPI,  but  only  12%  (n  =  88)  could  comment 
on  their  operators’  scores.  Of  those  88  SOF  leaders,  68%  (n  =  60)  reported  they  pay  attention  to  SOF 
operator  OPI  ratings;  56%  (n  =  49)  indicated  that  SOF  operator  OPI  ratings  were  important  or  very 
important,  and  44%  (n  =  38)  often  or  very  often  encouraged  SOF  operators  to  study  and  do  well  on  the 
OPI.  However,  the  small  number  of  SOF  leaders  who  indicated  they  could  respond  to  these  items  limits 
the  extent  to  which  these  findings  generalize  to  other  SOF  leaders. 


9  When  referring  to  SOF  operators  and  leaders,  this  section  focuses  only  on  the  SOF  operators  and  leaders  who 
participated  in  the  survey  and  were  either  familiar  with  or  could  comment  on  the  OPI.  Please  see  Appendix  B 
(Methodology)  and  the  Participation  Report  (Technical  Report  #2010011003)  for  more  information  about  survey 
respondents. 
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The  majority  of  SOF  operators  and  leaders  had  favorable  perceptions  of  the  OPI,  and  SOF  leaders  had 
slightly  more  favorable  perceptions  than  SOF  operators.  Both  SOF  operators  and  leaders  view  the  OPI  as 
an  accurate  means  for  assessing  speaking  proficiency  and  as  related  to  what  operators  do  while  deployed. 
Moreover,  SOF  operators  and  leaders  who  supported  the  standard  change  agreed  more  strongly  that  the 
OPI  relates  to  what  SOF  operators  do  on  missions,  suggesting  support  for  the  standard  change  may  be 
driven  by  perceiving  the  OPI  as  a  job-relevant  assessment.  However,  44%  (n  =  214)  of  SOF  operators 
and  58%  ( n  =  50)  of  SOF  leaders  agreed  that  there  are  language  tasks  SOF  operators  need  to  perform 
while  deployed  that  are  not  covered  by  the  OPI,  and  many  of  these  respondents  indicated  that  the  test 
should  cover  military-related  topics  or  relate  more  closely  to  their  mission. 

Detailed  Findings 

SOF  Leader  OPI  Testing  Background 

Very  few  SOF  leaders  ( n  =  88)  were  in  a  position  to  comment,  on  their  unit’s  scores,  limiting  the 
generalizability  of  the  findings  reported  in  this  section.  While  59%  ( n  =  442)  of  SOF  leaders  were 
familiar  with  the  OPI,  only  12%  (n  =  88)  could  comment  (Figure  5,  p.  20). 

Figure  5.  SOF  Leader  OPI  Familiarity 


Yes,  and  I  am  in  a 
position  to 
comment  on  my 
units  OPI  scores 
12% 


SOF  Leader  Uses  of  OPI  Scores 

Of  the  SOF  leaders  familiar  with  the  OPI  and  who  could  comment  (n  =  88),  68%  (n  =  60)  reported  paying 
attention  to  SOF  operator  OPI  ratings.  SOF  operator  OPI  ratings  were  important  or  very  important  to 
56%  (n  =  49)  of  SOF  leaders,  and  45%  (n  =  38)  of  SOF  leaders  often  or  very  often  encouraged  SOF 
operators  to  study  for  and  do  well  on  the  test.  Moreover,  those  SOF  leaders  that  pay  attention  indicated 
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that  OPI  ratings  are  more  important  (Figure  6,  p.  21)  and  encouraged  SOF  operators  to  study  and  do  well 
more  often  (Figure  7,  p.  21)  than  SOF  leaders  who  reported  not  paying  attention. 


Figure  6.  Importance  of  OPI  Ratings  to  SOF  Leaders 


60% 


Not  Important  Slightly  Moderately  Important  Very 

Important  Important  Important 


■  Pays  Attention 

■  Does  not  Pay  Attention 


Note.  Pays  Attention  =  leaders  who  indicated  they  pay  attention  to  operators’  OPI  ratings,  Does  not  Pay  Attention  =  leaders  who  indicated  they 
do  not  pay  attention  to  operators’  OPI  ratings. 


Figure  7.  SOF  Leader  Encouragement  to  Study  and  Do  Well  on  the  OPI 


40% 


Never  Seldom  Sometimes  Often  Very  Often 


Note.  Pays  Attention  =  leaders  who  indicated  they  pay  attention  to  operators’  OPI  ratings,  Does  not  Pay  Attention  =  leaders  who  indicated  they 
do  not  pay  attention  to  operators’  OPI  ratings. 


OPFs  Perceived  Job-relatedness  and  Accuracy 10 

SOF  operators  and  leaders  perceived  the  OPI  as  related  to  what  SOF  operators  do  on  the  job  and  during 
deployments  and  as  an  accurate  assessment  of  SOF  operators’  ability  to  use  language  on  the  job  and  on 

10  Appendices  D  (p.  34)  and  E  (p.  37)  provide  component  comparisons  for  how  SOF  operators  and  leaders  perceived 
the  OPI.  Appendix  F  (p.40)  provides  MI  linguist/09L  OPI  perceptions.  Appendix  G  (p.42)  presents  SOF  trainee 
perceptions. 
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mission-related  tasks.  Both  groups  agreed  that  the  OPI  was  moderately  related  to  very  related  to  job 
requirements  (Figure  8,  p.  22),  with  SOF  leaders  reporting  higher  levels  of  relatedness  than  SOF 
operators. 

For  both  groups,  those  who  supported  the  standard  change  more  often  reported  that  the  OPI  relates  to 
SOF  operator  job  requirements  and  accurately  assesses  SOF  operator  language  proficiency.  Specifically, 
SOF  operators  and  leaders  who  supported  the  standard  change  had  higher  agreement  that  the  OPI 
accurately  reflects  their  ability  to  use  the  language  on  the  job  and  that  the  content  of  the  OPI  is  related  to 
what  operators  do  during  deployments.11 


Figure  8.  Perceptions  of  OPI  Relatedness  to  Operators’  Jobs 


Not  Related  Slightly  Moderately  Related  Very  Related 
Related  Related 


■  SOF  Operators 

■  SOF  Leaders 


Comments  clarified  SOF  operator  and  leader  perceptions  of  the  OPI  (Table  5,  p.  23).  Many  SOF 
operators  and  leaders  reported  that  the  OPI  was  a  good  gauge  of  language  proficiency  because  it  assesses 
speaking  proficiency  through  conversation.  Focus  group  participants  also  commented  on  the  OPI’s 
capacity  for  assessing  speaking  proficiency  through  a  conversation  (Table  6,  p.  23): 


“This  is  a  better  way  to  measure  one  language  capability  by  active  conversation,  but  I 
realize  that  this  would  be  more  time  consuming.  ” 

SOF  Operator,  7th  SFG 


“Flow  do  we  all  really  communicate  in  real  life?  I  didn ’t  get  that.  You  don ’t  just  go  on 
and  talk  about  something  else.  We  repeat.  We  interact.  We  clarify.  That ’s  one  of  the 
benefits  of  that  telephone  test  that  they  used  to  do  because  that  way  a  real  human  could 
actually  assess  what  you  knew,  what  you  were  learning.  ’’ 

SOF  Operator,  5th  SFG  focus  group 

Even  though  SOF  operators  and  leaders  reported  favorable  job-re latedness  perceptions  of  the  OPI,  many 
operators  (44%,  n  =  214)  and  leaders  (58%,  n  =  50)  agreed  that  there  are  language  tasks  SOF  operators 


11  Appendices  H  (p.  44)  and  I  (p.  47)  provide  SOF  operator  and  leader  OPI  perceptions  compared  to  their  support  for 
the  standard  change. 
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need  to  perform  while  deployed  not  covered  by  the  OPI.  One  commonly  mentioned  criticism  by  this 
group  is  that  the  test  lacks  military -related  scenarios: 

“As  a  Civil  Affairs  Soldier  you  have  to  get  information  in  all  aspect  of  the  country  and 
OPI  does  not  cover  that  (Political,  Agricultural  etc..)-  ” 

SOF  Operator,  95th  CAB 


Table  5.  OPI  Perception  Comment  Themes 


SOF  Operators 

SOF  Leaders 

None/NA/taken  too  long  ago 

49 

7 

Good  gauge  of  language  proficiency/ability  to  communicate 

22 

14 

Should  cover  military  related  topics  or  be  related  to  the  mission 

12 

2 

Emphasis  on  the  OPI 

3 

4 

OPI  wasn't  relevant  (wrong  modality,  etc) 

4 

1 

Need  more  monetary  incentives  for  OPI 

0 

6 

Too  subjective 

0 

2 

Not  effective  for  reading  needs  or  not  good  replacement  for  DLPT 

3 

2 

Needs  to  be  done  face-to-face 

3 

1 

Cannot  use  dialect 

2 

1 

Not  mandatory 

1 

2 

Problem  with  Navy  requiring  baseline  DLPT  levels  before  taking  OPI 

0 

1 

Table  6.  Focus  Group  OPI  Themes 


Comments 

Other  positive  OPI  comments 

6 

OPI  is  a  good  indicator  of  proficiency 

5 

Other  negative  OPI  comments 

2 

OPI  content  is  related  to  mission/job/military 

1 
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SECTION  V:  CONCLUSIONS 

Overall,  this  report  contains  many  positive  findings  related  to  the  standard  change  and  Oral  Proficiency 
Interview  (  OPI)  testing.  However,  SOF  operators  and  leaders  identified  some  potential  limitations  of  the 
standard  change  and  OPI  testing  that  may  limit  their  acceptance  and  support.  This  section  provides  the 
key  conclusions  based  on  feedback  provided  by  survey  respondents  and  focus  group  participants. 

Reactions  to  the  Standard  Change 

Overall,  many  SOF  operators  and  leaders  supported  the  standard  change  and  very  few  opposed  it. 

Despite  these  positive  reactions  to  the  standard  change,  some  SOF  operators  and  leaders  reported  they 
were  neutral  (neither  opposed  nor  supported)  regarding  the  standard  change.  Based  on  their  comments, 
three  possible  reasons  emerged  for  these  neutral  and  opposing  opinions: 

•  First,  many  respondents  commented  that  reading  is  still  an  important  skill  for  mission  success  and 
should  be  included  in  language  testing.  This  suggests  that  a  reading  test  may  be  necessary  for 
some  SOF  operators. 

•  Second,  some  were  concerned  that  SOF  operator  language  training  and  language  proficiency 
incentive  programs  had  not  been  updated  to  align  with  OPI  testing,  so  SOF  operators  would  not 
be  prepared  for  OPI  testing  nor  would  they  be  motivated  to  take  it.  Thus,  ensuring  SOF  operators 
are  prepared  for  the  OPI  and  paid  for  OPI  scores  at  the  appropriate  levels  should  be  addressed 
within  all  components.  Communicating  how  the  OPI  is  used  to  pay  Foreign  Language 
Proficiency  Bonus  -  currently  and  as  policy  changes  -  will  be  key  to  addressing  this  concern  for 
SOF  personnel. 

•  Third,  unfamiliarity  with  the  change  and/or  inexperience  with  OPI  testing,  particularly  two-skill 
OPI  testing,  were  mentioned  as  reasons  why  SOF  operators  and  leaders  were  undecided  about  the 
standard  change.  This  suggests  that  information  about  the  standard  change  and  associated  testing 
should  be  widely  disseminated. 

Operator  OPI  Testing  Background 

Overall,  44%  ( n  =  505)  of  SOF  operators  reported  having  taken  an  official  OPI  and  77%  (n  =  448)  of 
these  SOF  operators  have  not  encountered  problems  during  testing.  Of  those  few  SOF  operators  who 
experienced  problems,  issues  included  disruptions  while  testing,  delays/problems  receiving  feedback  on 
their  score,  and  technical  problems  (e.g.,  not  being  able  to  hear  the  tester  on  the  phone).  Disruptions 
suggest  a  lack  of  sufficient  control  and/or  standardization  in  the  testing  environments  or  that  facilities  are 
not  structured  for  OPI  testing.  This  should  be  investigated.  These  findings  suggest  the  OPI  often 
functions  properly,  but  the  occasional  problems  that  do  arise  should  be  addressed  as  they  are  likely  to 
become  more  frequent  as  OPI  testing  increases  with  the  standard  change. 
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OPI  Testing  Perceptions 

While  more  than  half  of  SOF  leaders  who  responded  were  familiar  with  the  OPI,  only  12%  (n  =  88)  could 
comment  on  their  operators’  scores.  Of  these  SOF  leaders  (n  =  88),  68%  indicated  they  pay  attention  to 
SOF  operator  OPI  ratings;  56%  indicated  that  SOF  operator  OPI  ratings  were  important  or  very 
important,  and  44%  often  or  very  often  encouraged  SOF  operators  to  study  and  do  well  on  the  OPI.  This 
may  be  a  result  of  SOF  leaders’  unfamiliarity  with  the  standard  change.  As  OPI  testing  continues  to 
increase,  the  amount  of  attention  and  importance  SOF  leaders  place  on  OPI  scores  should  be  monitored  to 
see  if  they  also  increase. 

Many  SOF  operators  and  leaders  had  favorable  perceptions  of  the  OPI’s  job-re latedness  and  accuracy. 
Both  view  the  OPI  as  an  accurate  means  for  assessing  speaking  proficiency  and  as  related  to  what  SOF 
operators  do  while  deployed.  Moreover,  SOF  operators  and  leaders  who  supported  the  standard  change 
agreed  more  strongly  that  the  OPI  accurately  assesses  and  relates  to  what  SOF  operators  do  on  missions. 
However,  some  SOF  operators  and  leaders  agreed  that  there  are  language  tasks  SOF  operators  need  to 
perform  while  deployed  that  are  not  covered  by  the  OPI.  Additionally,  many  of  these  respondents 
indicated  that  the  test  should  cover  military-related  topics  or  relate  more  closely  to  their  mission.  Since 
the  OPI  is  a  measure  of  functional  speaking  proficiency  (and  participatory  listening  proficiency  in  the 
case  of  the  two-skill  OPI),  more  mission-related  prompts  and  role  plays  could  be  introduced  into  the  OPI 
without  compromising  the  assessment  of  proficiency  tasks.  However,  this  would  require  a  modification 
of  the  assessment  and  retraining  of  the  OPI  testers. 
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ABOUT  SWA  CONSULTING  INC. 

SWA  Consulting  Inc.  (formerly  Surface,  Ward,  and  Associates)  provides  analytics  and  evidence-based 
solutions  for  clients  using  the  principles  and  methods  of  industrial/organizational  (I/O)  psychology.  Since 
1997,  SWA  has  advised  and  assisted  coiporate,  non-profit  and  governmental  clients  on: 

•  Training  and  development 

•  Performance  measurement  and  management 

•  Organizational  effectiveness 

•  Test  development  and  validation 

•  Program/training  evaluation 

•  Work/job  analysis 

•  Needs  assessment 

•  Selection  system  design 

•  Study  and  analysis  related  to  human  capital  issues 

•  Metric  development  and  data  collection 

•  Advanced  data  analysis 

One  specific  practice  area  is  analytics,  research,  and  consulting  on  foreign  language  and  culture  in  work 
contexts.  In  this  area,  SWA  has  conducted  numerous  projects,  including  language  assessment  validation 
and  psychometric  research;  evaluations  of  language  training,  training  tools,  and  job  aids;  language  and 
culture  focused  needs  assessments  and  job  analysis;  and  advanced  analysis  of  language  research  data. 

Based  in  Raleigh,  NC,  and  led  by  Drs.  Eric  A.  Surface  and  Stephen  J.  Ward,  SWA  now  employs  close  to 
twenty  I/O  professionals  at  the  masters  and  PhD  levels.  SWA  professionals  are  committed  to  providing 
clients  the  best  data  and  analysis  upon  which  to  make  evidence-based  decisions.  Taking  a  scientist- 
practitioner  perspective,  SWA  professionals  conduct  model-based,  evidence-driven  research  and 
consulting  to  provide  the  best  answers  and  solutions  to  enhance  our  clients’  mission  and  business 
objectives.  SWA  has  competencies  in  measurement,  data  collection,  analytics,  data  modeling,  systematic 
reviews,  validation,  and  evaluation. 

For  more  information  about  SWA,  our  projects,  and  our  capabilities,  please  visit  our  website  (www.swa- 
consulting.com)  or  contact  Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr.  Stephen  J.  Ward 
(sward@swa-consulting.com). 

The  SWA  Consulting  Inc.  team  members  contributed  to  this  report  (listed  in  alphabetical  order): 

Ms.  Sarah  C.  Bienkowski 

Mr.  Milton  V.  Cahoon 

Dr.  Reanna  Poncheri  Harman 

Mr.  Ryan  Phillips 

Dr.  Eric  A.  Surface 

Dr.  Stephen  J.  Ward 

Ms.  Sheila  L.  Wilcox 

Ms.  Natalie  Wright 
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APPENDIX  A:  ABOUT  THE  LCNA  PROJECT 

In  2003-2004,  the  Special  Operations  Forces  Language  In  2003-2004,  the  Special  Operations  Forces 
Language  Office  (SOFLO)  sponsored  the  SOF  Language  Transformation  Strategy’  Needs  Assessment 
Project  to  inform  the  development  of  a  language  transformation  strategy  in  response  to  a  GAO  report 
(2003).  This  SOF  Language  Transformation  Strategy’  Needs  Assessment  Project  collected  current-state 
information  about  language  usage,  proficiency,  training,  and  policy  issues  (e.g.,  Foreign  Language 
Proficiency  Pay,  FLPP)  from  SOF  personnel,  SOF  unit  leaders,  and  other  personnel  involved  in  SOF 
language.  The  project  used  multiple  data  collection  methods  and  provided  the  SOFLO  with  valid  data  to 
develop  a  comprehensive  language  transformation  strategy  and  advocate  for  the  SOF  perspective  on 
language  issues  within  the  DoD  community. 

In  a  continuing  effort  to  update  knowledge  of  language  and  culture  needs  while  informing  strategic  plan 
development,  the  SOFLO  commissioned  the  2009  SOF  Language  and  Culture  Needs  Assessment  Project 
(LCNA)  to  reassess  the  language  and  culture  landscape  across  the  United  States  Special  Operations 
Command  (USSOCOM)  and  develop  a  strategy  for  the  next  five  years.  Data  were  collected  between 
March  and  November,  2009  from  personnel  in  the  SOF  community,  including  SOF  operators  and  leaders. 
Twenty-three  focus  groups  were  conducted  between  March  and  June,  2009.  A  comprehensive,  web-based 
survey  for  SOF  operators  and  leaders  was  launched  on  26  October  and  closed  on  24  November,  2009. 

This  project’s  findings  will  be  disseminated  through  reports  and  briefings  (see  Appendix  A,  Figure  1). 
Two  foundational  reports  document  the  methodology  and  participants  associated  with  this  project.  The 
remaining  reports  are  organized  in  three  tiers.  Twenty-five  Tier  I  reports  focus  on  specific,  limited  issues 
(e.g.,  Inside  AOR  Use  of  Language).  Tier  II  reports  integrate  and  present  the  most  important  findings 
across  related  Tier  /  reports  (e.g.,  Use  of  Language  and  Culture  on  Deployment)  while  including 
additional  data  and  analysis  on  the  topic.  Most,  but  not  all,  Tier  I  reports  will  roll  into  Tier  //  reports.  One 
Tier  III  report  presents  the  most  important  findings,  implications,  and  recommendations  across  all  topics 
explored  in  this  project.  The  remaining  Tier  III  reports  present  findings  for  specific  SOF  organizations 
[e.g.,  Air  Force  Special  Operations  Command  (AFSOC),  Special  Forces  (SF)  Command].  All  Tier  III 
reports  are  associated  with  a  briefing.  Report  topics  are  determined  by  the  SOFLO  and  subject  to  change. 

In  June,  2009,  the  GAO  reported  that  the  Department  of  Defense  is  making  progress  toward  transforming 
language  and  regional  proficiency  capabilities  but  still  does  not  have  a  strategic  plan  in  place  to  continue 
development  that  includes  actionable  goals  and  objectives.  The  findings  from  this  study  can  be  used  by 
the  SOFLO  and  leaders  at  USSOCOM  to  continue  strategic  planning  and  development  in  this  area. 

This  project  design,  logistics,  data  collection,  initial  analysis  and  first  eight  reports  of  this  project  were 
conducted  by  SWA  Consulting  Inc.  (SWA)  under  a  subcontract  with  SRC  (SR20080668  (K142);  Prime  # 
N65236-08-D-6805).  The  additional  reports  are  funded  under  a  separate  contracting  vehicle  with  Gemini 
Industries  Inc.  [GEM02-ALMBOS-0018  (10210SWA-1);  Prime  #  USZA22-02-D-0015],  For  questions  or 
more  information  about  the  SOFLO  and  this  project,  please  contact  Mr.  Jack  Donnelly 
(iohn.donnelly@socom.mil).  For  specific  questions  related  to  data  collection  or  reports  associated  with 
this  project,  please  contact  Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr.  Reanna  Poncheri 
Harman  (ipharman@swa-consulting.com)  with  SWA  Consulting  Inc. 
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Appendix  A,  Figure  1.  Report  Overview 


Foundation  Reports 


1.  Methodology  Report 

2.  Participation  Report 


Tier  I  Reports  First  Contract 


3.  Reactions  to  Admiral  Olson's  Memo 

4.  Training  Emphasis:  Language  and  Culture 

5.  Command  Support:  Grading  the  Chain  of 
Command 

6.  SOFLO  Support 

7.  Inside/Outside  AOR  Use  of  Cultural  Knowledge 

8.  Team  Composition 


Tier  I  Reports  Second  Contract 


9.  Inside  AOR  Use  of  Language 

10.  Outside  AOR  Use  of  Language 

11.  Mission-Specific  Use  of  Interpreters 

12.  General  Use  of  Interpreters 

13.  09L  Use  in  the  Special  Operations  Forces 
Community 

14.  DLPT 

15.  OPI 

16.  DLAB:  Perspectives  from  the  Field 

17.  Initial  Acquisition  Training 

18.  Sustainment/Enhancement  Training 

19.  Culture  Training 

20.  Immersion  Training 

21.  Language  Resources,  Technology  8t  Self-Study 

22.  Foreign  Language  Proficiency  Bonus 

23.  Non-monetary  Incentives 

24.  Considering  Language  in  the  Promotion  Process 

25.  Barriers  to  Language  Acquisition  and 
Maintenance 

26.  Force  Motivation  for  Language 

27.  Leader  Perspectives  on  Language  Issues 

28.  CLPM  Perspectives 


Tier  II  Reports  Second  Contract 


29.  Use  of  Language  and  Culture  on  Deployment 

30.  Use  of  Interpreters 

31.  Team  Composition  and  Capability 

32.  Testing/Metrics 

33.  Current  State  of  Language  Training 

34.  Language  Training  Guidance 

35.  Culture  Training  Guidance 

36.  Incentives/Barriers 

I _ 


Tier  III  Reports  Second  Contract 


37.  Overall  Picture:  Conclusions  and 
Recommendations 

38.  AFSOC 

39.  MARSOC 

40.  WARCOM 

41.  SF  Command 

42.  CA 

43.  PSYOP 

44.  Seminar  Briefing(s) 


Note'.  Foundation  reports  are  referenced  by  every  other  report.  Colors  represent  Tier  I  reports  that  roll  (integrate)  into  an  associated  Tier  II  report.  Reports  in  black  are  final  reports  on  the  topic 
but  may  be  cited  by  other  reports.  Tier  II  reports  roll  into  the  Tier  III  reports.  All  Tier  III  reports  include  an  associated  briefing. 
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APPENDIX  B:  METHODOLOGY 

Participants 

Focus  Group 

Twenty-three  focus  groups  were  conducted  with  126  SOF  personnel  across  the  SOF  community.  Focus 
groups  were  conducted  with  Air  Force  Special  Operations  Command  (AFSOC),  Marine  Corps  Forces 
Special  Operations  Command  (MARSOC),  Navy  Special  Warfare  Command  (WARCOM),  and  United 
States  Army  Special  Operations  Command  (USASOC).12  Comments  from  focus  group  participants 
regarding  the  OPI  are  included  throughout  this  report. 

Survey 

Survey  respondents  received  the  operator  version  of  the  OPI  and  standard  change  items  if  they  indicated 
one  of  the  following  SOF  community  roles: 

•  SOF  Operator 

•  SOF  Operator  assigned  to  other  duty 

•  Currently  in  the  training  pipeline  for  SOF 

•  Military  Intelligence  (MI)  Linguist  or  09L  assigned  or  attached  to  a  SOF  unit 

•  SOF  Retirees 

Respondents  received  the  leader  version  of  the  OPI  and  standard  change  items  if  they  designated  one  of 
the  following  SOF  community  roles: 

•  SOF  Unit  Commanders  and  Unit  Leadership  of  03  Commands  or  higher,  including  Staff, 
Support,  and  Specialists 

•  Command  Language  Program  Manager  or  Component  Language  Program  Managers  (CLPMs) 

•  Language  Office  Personnel 

•  Instructor 

Throughout  this  report,  “SOF  operator”  refers  to  respondents  who  indicated  they  were  a  SOF  Operator  or 
a  SOF  Operator  assigned  to  other  duty.  The  “SOF  leader”  responses  throughout  this  report  only  include 
SOF  Unit  Commanders  and  Unit  Leadership  of  03  Commands  or  higher.  In  addition  for  comparisons, 
responses  from  “MI  Linguists  or  09Ls  assigned  or  attached  to  SOF  unit”  are  provided  in  Appendix  F  and 
Appendix  G  provides  responses  from  operators  currently  in  a  training  pipeline. 

SOF  Retirees  were  excluded  from  this  report  because  of  the  small  number  that  had  taken  an  official  OPI. 
Command  Language  Program  Manager  or  Component  Language  Program  Managers  (CLPMs),  Language 
Office  Personnel,  and  Instructors  were  excluded  from  this  report  because  of  the  small  number  who  said 
they  were  in  a  position  to  comment  on  the  OPI. 


12  See  Participation  Report  (Technical  Report  #201001 1003)  for  participant  details.  See  Methodology’  Report 
(Technical  Report  #2010011002)  for  the  focus  group  interview  guide. 
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SOF  Type  Classification 

SOF  operators  and  leaders  assigned  to  the  USASOC  were  separated  according  to  their  SOF  type  in 
Appendices  C  through  F. 15  Respondents  from  USASOC  were  classified  in  SOF  types  [i.e.,  Civil  Affairs 
(CA),  Military  Information  Support  Group  (MISG),  and  Special  Forces  (SF)]  based  on  the  following 
criteria: 

•  USASOC  unit  to  which  they  were  assigned  (e.g.,  1st  SFG  classified  as  SF,  4th  POG  as  MISG,  95th 
CAB  as  CA) 

•  Reported  MOS  (e.g.,  18  series  were  classified  as  SF,  37  series  as  MISG,  38  series  as  CA) 

Measures 

Survey  respondents  received  closed-ended  and  open-ended  items  related  to  the  OPI.  Whether  SOF 
operators  and  leaders  were  presented  these  items  depended  on  their  response  to  the  following  items: 

SOF  Operators 

•  Have  you  taken  an  official  Oral  Proficiency  Interview  (OPI)  in  your  current  official  or  required 
language  or  your  primary/control  language? 

Respondents  that  indicated  “Yes,  I  have  taken  an  official  OPI”  (44%,  n  =  505)  received  the  OPI  follow-up 
items.  Respondents  that  indicated  “No,  I  have  only  taken  an  unofficial  OPE”  (20%,  n  =  23 1)  or  “No,  I 
have  never  taken  a  speaking  test  in  my  current  official  or  required  AOR  language”  (36%,  n  =  408)  were 
branched  to  the  next  survey  section. 

SOF  Leaders 

•  Are  you  familiar  with  official  Oral  Proficiency  Interviews  (OPIs)? 

Respondents  that  indicated  “Yes,  and  I  am  in  a  position  to  comment  on  my  unit’s  OPI  scores”  (12%,  n  = 
88)  received  the  OPI  follow-up  items.  Respondents  that  indicated  “Yes,  but  I  am  not  in  a  position  to 
comment  on  my  unit’s  OPI  scores”  (47%,  n  =  354)  or  “No,  I  am  not  familiar  with  the  OPI”  (41%,  n  = 

303)  were  branched  to  the  next  survey  section. 

OPI  and  Standard  Change  Items 

SOF  operators  who  had  taken  an  official  OPI  in  their  current  official  or  required  language  were  asked 
about  any  technical  problems  experienced  during  their  last  testing  event,  as  well  as  their  perceptions  of 
the  OPI’s  relatedness  to  their  jobs  and  accuracy  in  assessing  their  language  skills.  SOF  leaders  who  were 
familiar  with  the  OPI  and  could  comment  on  it  were  asked  about  how  much  they  pay  attention  to  OPI 
ratings,  how  important  the  ratings  are  to  them,  and  how  much  they  encourage  their  operators  to  study  and 


13 

For  further  details  on  participation  and  attrition  rates,  please  refer  to  the  Participation  Report  (Technical  Report 
#2010011003). 
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do  well  on  the  test.  They  also  responded  to  a  series  of  items  that  assessed  their  perceptions  of  the  OPI’s 
relatedness  to  operators’  jobs,  and  accuracy  in  assessing  operator  language  ability. 

Standard  Change  Items 

SOF  leaders  and  operators  responded  to  one  closed-ended  and  one  open-ended  item  regarding  the 
standard  change,  regardless  of  whether  they  were  familiar  with  or  were  able  to  comment  on  the  OPI. 

Analyses 

All  closed-ended  items  were  analyzed  using  a  combination  of  descriptive  and  inferential  statistics.  For 
select  items,  frequencies  and  average  (i.e.,  mean)  responses  for  each  item  are  presented  in  Appendices  C 
through  E.  To  compare  responses  across  groups  of  participants  (e.g.,  leaders  v.  operators),  inferential 
statistics  (e.g.,  analysis  of  variance,  t-tests)  were  used  to  determine  if  any  observed  differences  are  likely 
to  exist  in  the  broader  population  of  interest. 

For  qualitative  data  (i.e.,  open-ended  comments),  raters  created  a  content  code  (i.e.,  theme)  list  based  on 
available  responses  for  each  item.  A  primary  rater  then  coded  each  response  and  a  secondary  rater  coded 
30%  of  the  responses.  Raters  determined  the  consistency  of  codes  and  discussed  any  disagreements  to 
consensus.  Frequency  of  occurrence  for  each  theme  is  presented  in  Sections  II  through  IV.  A  similar 
process  was  used  to  code  the  focus  group  data.14  Two  different  coders  from  the  survey  comments  rated 
the  content  of  each  focus  group  segment.  Totals  for  themes  and  verbatim  comments  related  to  OPI  testing 
and/or  the  standard  change  are  provided  throughout  the  report. 


14  For  further  details  on  these  methods,  please  refer  to  the  Methodology  Report  (Technical  Report  #2010011002). 
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APPENDIX  C:  REACTIONS  TO  THE  STANDARD  CHANGE 

Appendix  C,  Table  1.  SOF  Operators  -  The  test  of  record  for  SOCOM  has  changed  from  the  DLPT,  which  assesses  listening  and  reading,  to  a  two- 
skill  OPI,  which  assesses  listening  and  speaking.  What  is  your  reaction  to  this  change? 


Group 

n 

M 

Strongly  Oppose 

Somewhat  Oppose 

Neither  Oppose  nor 
Support 

Somewhat  Support 

Strongly  Support 

Overall 

1,039 

3.55 

5%i 

6%l 

45%  1  1 

21%  l 

24%  t 

AFSOC 

19 

3.32 

11%  ■ 

0%l 

47%  [  ! 

32%  l 

1 1%  [ 

WARCOM 

8 

4.37 

0%l 

0%I 

25%  1 

13%  1 

63%  r 

MARSOC 

12 

3.75 

8%[ 

0%l 

33%  1 

25%  I 

33%  I 

USASOC 

751 

3.53 

5%  I 

7%l 

44%  1 

20%  1 

24%  1 

CA 

149 

3.63 

4%  I 

6%l 

40% 

22%  j 

28% 

MISG 

124 

3.41 

9%  I 

9%i 

39% 

19%  1 

24% ! 

SF 

471 

3.52 

4%  I 

6%l 

48% 

20%  1 

23%  I 

Note.  1  =  Strongly  Oppose ,  2  =  Somewhat  Oppose ,  3  =  Neither  Oppose  nor  Support,  4  =  Somewhat  Support,  5  =  Strongly  Support.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table. 
There  were  no  significant  group  differences  between  components  or  Army  SOF  types. 


Appendix  C,  Table  2.  SOF  Leaders  -  The  test  of  record  for  SOCOM  has  changed  from  the  DLPT  which  assesses  listening  and  reading  to  a  two-skill 
OPI,  which  assesses  listening  and  speaking.  What  is  your  reaction  to  this  change? 


Neither  Oppose  nor 


Group 

n 

M 

Strongly  Oppose 

Somewhat  Oppose 

Support 

Somewhat  Support 

Strongly  Support 

Overall 

800 

3.70 

3%l 

6%l 

37%  1 

27% 

27%  t 

AFSOC 

9 

3.33 

11%I 

0%l 

44%  [ 

33% 

1 1%  1 

WARCOM 

11 

4.27 

0%I 

0%I 

27%  1 

18% 

55%  [ 

MARSOC 

22 

4.05 

0%I 

5%  I 

32%  1 

18% 

45%  [ 

USASOC 

488 

3.70 

3%l 

7%  I 

34%  I 

28% 

28%  1 

CA 

63 

3.67 

3%  II 

5%l 

40% 

27% 

25% 

MISG 

95 

3.61 

3%  II 

7%i 

36% 

33% 

21% 

SF 

253 

3.76 

3%  II 

7% 

33% 

26% 

32% 

Note.  1  =  Strongly  Oppose,  2  =  Somewhat  Oppose,  3  =  Neither  Oppose  nor  Support,  4  =  Somewhat  Support,  5  =  Strongly  Support.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table. 
There  were  no  significant  group  differences  between  components  or  Army  SOF  types. 
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APPENDIX  D:  SOF  OPERATOR  PERCEPTIONS  OF  THE  OPI 

Appendix  D,  Table  1.  How  related  is  the  OPI  to  what  you  do  on  the  job? 


Group 

n 

M 

Not  Related 

Slightly  Related 

Moderately  Related 

Related 

Very  Related 

Overall 

495 

3.02 

16%  I 

18%  I 

29% 

21%  I 

16% 

AFSOC 

6 

3.67 

0%l 

0%l 

50% 

33%  I 

17% 

WARCOM 

3 

2.67 

33%  1 

0%l 

33% 

33%  1 

0%l 

MARSOC 

3 

4.00 

0%l 

0%[ 

33% 

33%  1 

33% 

USASOC 

369 

2.98 

18%  1 

18%  1 

28% 

20%  1 

16% 

CA 

63 

3.57b 

OO 

is 

— 

12% 

26% 

23% 

31% 

MISG 

91 

3.02a 

14% 

19% 

33% 

17% 

16% 

SF 

207 

2.78a 

23% 

19% 

26% 

20%  [ 

11% 

Note.  1  =  Not  Related,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  Army  SOF  types  sharing  the 
same  letter  (e.g.,  a  or  b)  did  not  report  significantly  different  responses.  There  were  no  significant  differences  between  components. 


Appendix  D,  Table  2.  The  content  of  the  OPI  is  clearly  related  to  what  I  do  during  deployment. 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree 

Dis  agree 

Disagree 

Agree 

Strongly  Agree 

Overall 

484 

3.02 

12%  1 

13%  1 

43% 

25%  [ 

7%  I 

AFSOC 

6 

4.00 

0%l 

0%l 

33% ! 

33% 

33%  [ 

WARCOM 

3 

3.33 

0%l 

0%l 

67%  I 

33%  [ 

0%l 

MARSOC 

2 

3.50 

0%l 

0%l 

50% 

50% 

0%l 

USASOC 

361 

2.97 

14%  I 

14% 

41%  | 

24% 

7%l 

CA 

63 

3.59b 

3%l 

5% 

38% 

38% 

16% 

MISG 

91 

2.95a 

13%  1 

11% 

49% 

21% 

5%  I 

SF 

207 

2.79a 

17% 

18% 

39% 

21% 

5%l 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  Army  SOF  types 
sharing  the  same  letter  (e.g.,  a  or  b)  did  not  report  significantly  different  responses.  There  were  no  significant  differences  between  components. 
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Appendix  D,  Table  3.  My  OPI  rating  accurately  reflects  my  ability  to  use  language  while  on  the  job. 


Group 

n 

M 

Strongly  Disagree 

Neither  Agree  nor 

Disagree  Disagree 

Agree  Strongly  Agree 

Overall 

483 

3.20 

8% 

12% 

39% 

34% 

7% 

AFSOC 

6 

4.501 

0% 

0% 

0% 

50%  50%  1 

WARCOM 

3 

3.671’2 

0% 

0% 

33% 

67% 

0%l 

MARSOC 

3 

3.671'2 

0% 

0% 

67% 

0% 

33% 

USASOC 

358 

3.162 

9% 

13% 

37% 

33% 

7% 

CA 

63 

3.43b 

0% 

19% 

32% 

37% 

13% 

MISG 

91 

3.23ab 

10% 

8% 

38% 

37% 

7% 

SF 

204 

3.04a 

12% 

14% 

38% 

30% 

6% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  =  Neither  Agree  nor  Disagree ,  4  =  Agree ,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  Army  SOF  types 
sharing  the  same  letter  (e.g.,  a  or  b)  did  not  report  significantly  different  responses.  Components  sharing  the  same  number  (e.g.,  1  or  2)  did  not  report  significantly  different  responses. 


Appendix  D,  Table  4.  Operators  who  perform  well  on  the  OPI  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  perform 
poorly  on  the  OPI. 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

484 

3.43 

6% 

7% 

38% 

33% 

15% 

AFSOC 

6 

4.50 

0% 

0% 

17% 

17% 

67%  [ 

WARCOM 

3 

4.33 

0% 

0% 

33% 

67%  ■  0% 

MARSOC 

3 

3.67 

0% 

0% 

67% 

0% 

33% 

USASOC 

359 

3.40 

8% 

8% 

36% 

33% 

15% 

CA 

63 

3.73a 

2% 

6% 

30% 

41% 

21% 

MISG 

91 

3.37ab 

11% 

4% 

35% 

35% 

14% 

SF 

205 

3.30b 

8% 

10% 

38% 

30% 

14% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  Army  SOF  types 
sharing  the  same  letter  (e.g.,  a  or  b)  did  not  report  significantly  different  responses.  There  were  no  significant  differences  between  components. 
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Appendix  D,  Table  5.  My  OPI  rating  is  an  accurate  reflection  of  my  ability  to  perform  job/mission-related  tasks  in  the  target  language. 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Overall 

485 

3.13 

8%l 

15%  1 

42%  1 

26%  1 

9%l 

AFSOC 

6 

4.33a 

0%[ 

0% 

0% 

67%  E 

33%  1 

WARCOM 

3 

4.00ab 

0%[ 

0% 

33%, 

33%  1 

33%  1 

MARSOC 

3 

3.33ab 

0%l 

33% 

33% 

0%' 

33% 

USASOC 

360 

3.05b 

9%l 

17%  [ 

43% 

24%  [ 

v? 

ox 

OO 

CA 

63 

3.32 

2%[ 

22% 

32% 

32% 

13% 

MISG 

91 

3.03 

11% 

11% 

48% 

23%  1 

7%  I 

SF 

206 

2.98 

10%  I 

17% 

43% 

22% 

7%  II 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  =  Neither  Agree  nor  Disagree ,  4  =  Agree ,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  Components  sharing 
the  same  letter  (e.g.,  a  or  b)  did  not  report  significantly  different  responses.  There  were  no  significant  differences  between  Army  SOF  types. 


Appendix  D,  Table  6.  There  are  language  tasks  I  need  to  perform  when  deployed  that  are  not  covered  by  the  OPI. 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Dis  agree 

Agree 

Strongly  Agree 

Overall 

485 

3.40 

5%  I 

8% 

43% 

30% 

14%  1 

AFSOC 

6 

4.17 

0%l 

0%I 

17%  [ 

50% 

33%  1 

WARCOM 

3 

3.33 

0%l 

33%. 

33%  I 

0%l 

33%  1 

MARSOC 

3 

4.00 

0%l 

0%I 

33% 

33% 

33%  1 

USASOC 

360 

3.41 

6%  I 

9%l 

41%  I 

29% 

16%  I 

CA 

63 

3.54 

2%  1 1 

13% 

37% 

29% 

21%  I 

MISG 

91 

3.33 

7%  II 

8%  I 

44% 

30% 

12% 

SF 

206 

3.41 

6%  II 

8%  I 

40% 

29% 

17% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  There  were  no 
significant  differences  between  components  or  Army  SOF  types. 
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APPENDIX  E:  SOF  LEADER  PERCEPTIONS  OF  THE  OPI 


Appendix  E,  Table  1.  How  related  is  the  OPI  to  what  operators  in  your  unit  do  on  the  job? 


Group 

n 

M 

Not  Related 

Slightly  Related 

Moderately  Related 

Related 

Very  Related 

Overall 

87 

3.64 

ll%i 

9%l 

17%T 

28%  ■ 

34%  1 

AFSOC 

1 

5.00 

o%| 

0%I 

0%I 

0%! 

100% 

WARCOM 

2 

5.00 

0%I 

0%I 

0%[ 

0%I 

100% 

MARSOC 

1 

4.00 

0%I 

0%I 

0%l 

100% 

0%I 

USASOC 

65 

3.54 

12%  [ 

11%  I 

18% 

28%  I 

31%  1 

CA 

6 

3.67 

0%l 

0%l 

50% 

33%  1 

17%  i 

MISG 

20 

3.35 

20% 

0%  II 

20% 

45% 

15%  1 

SF 

29 

3.52 

14% 

14%  1 

17% 

17% ! 

38% 

Note.  1  =  Not  Related,  2 

=  Slightly  Related ,  3 

=  Moderately  Related ,  4 

=  Related ,  5  =  Very  Related.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table. 

There  were  no  significant 

differences  between  components  or  Army  SOF  types. 


Appendix  E,  Table  2.  The  content  of  the  OPI  is  clearly  related  to  what  operators  do  during  deployment. 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Disagree 

Agree 

Strongly  Agree 

Overall 

86 

3.34 

7%1 

14%  [ 

33%  ■ 

3 1%  I 

15%  1 

AFSOC 

1 

4.00 

o%! 

0%I 

0%| 

100%  1 

0%l 

WARCOM 

2 

4.50 

0%|[ 

0%l 

0%l 

50% 

50%  I 

MARSOC 

1 

4.00 

0%|[ 

0%I 

0%l 

100% 

0%l 

USASOC 

64 

3.23 

8%[ 

16%  [ 

36%  1. 

27% 

14%  I 

CA 

6 

3.67 

0%l 

0%l 

50% 

33%  I 

17%  I 

MISG 

19 

3.26 

5%l 

L 1  %  I 

47% 

26% 

11% 

SF 

29 

2.97 

14%: 

21% 

31% 

24% 

10%  I 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  There  were  no 
significant  differences  between  components  or  Army  SOF  types. 
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Appendix  E,  Table  3.  An  operator's  OPI  rating  accurately  reflects  their  ability  to  use  language  while  on  the  job. 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree 
Disagree 

nor 

Agree 

Strongly  Agree 

Overall 

86 

3.56 

8%l 

6%  I 

26%  ■ 

43% 

17%H 

AFSOC 

1 

4.00 

0%[ 

0%[ 

0%l 

100% 

0% 

WARCOM 

2 

4.50 

0%l 

0%[ 

0%l 

50% 

50% 

MARSOC 

1 

4.00 

0%[ 

0%| 

0%l 

100% 

0% 

USASOC 

64 

3.48 

9%[ 

6% 

28%  1 

39% 

17% 

CA 

6 

4.17 

0%1 

0% 

17%  I 

50% 

33% 

MISG 

19 

3.53 

0%[ 

5% 

47%  1 

37% 

ii% 

SF 

29 

3.34 

14%  1 

7%l 

24%  [ 

41% 

14% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  = 
significant  differences  between  components  or 

:  Neither  Agree  nor  Disagree ,  4 
Army  SOF  types. 

=  Agree ,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  There  were  no 

Appendix  E,  Table  4.  Operators  who  perform  well  on  the  OPI  are  more 
poorly  on  the  OPI. 

likely  to  successfully  use  language  in  the  field  than  those  who  perform 

Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree 
Disagree 

nor 

Agree 

Strongly  Agree 

Overall 

86 

3.81 

3%1 

3%  I 

23%  ■ 

48% 

22%  H 

AFSOC 

1 

4.00 

0%[ 

0%[ 

0%l 

100% 

0% 

WARCOM 

2 

5.00 

0%l 

0%l 

0%l 

o%: 

100% 

MARSOC 

1 

4.00 

0%| 

0%[ 

0%l 

100% 

0% 

USASOC 

64 

3.75 

3%l 

3%l 

28%  1 

47% 

19% 

CA 

6 

3.50 

0%l 

0% 

50%  ■ 

50% 

0% 

MISG 

19 

3.79 

0%| 

0% 

32%  L 

58% 

11% 

SF 

29 

3.72 

7%  I 

3%  |[ 

24%  i 

41% 

24% 

Note.  1  =  Strongly  Disagree ,  2 

=  Disagree ,  3  = 

:  Neither  Agree  nor  Disagree ,  4  : 

=  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  There  were  no 

significant  differences  between  components  or  Army  SOF  types. 
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Appendix  E,  Table  5.  An  operator's  OPI  rating  is  an  accurate  reflection  of  their  ability  to  perform  job/mission-related  tasks  in  the  target  language. 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Dis  agree 

Agree 

Strongly  Agree 

Overall 

86 

3.52 

6%  I 

9%l 

28%  1 

41%  [ 

16%B 

AFSOC 

1 

4.00 

0%|[ 

0%l 

0%I 

100%  L 

0%l 

WARCOM 

2 

4.50 

0%I 

0%l 

0%l 

50%  [ 

50%  BT 

MARSOC 

1 

4.00 

o%! 

0%l 

0%| 

100%  [ 

0%l 

USASOC 

64 

3.45 

6%I 

9%l 

33%  I 

36% 

16%1 

CA 

6 

3.67 

0%[| 

0%l 

33% 

67% 

0%l 

MISG 

19 

3.32 

5%  1 

11% 

42% 

32% 

11%  I 

SF 

29 

3.45 

7%  I 

10%  I 

31% 

34%  ■ 

17%  1 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  = 
significant  differences  between  components  or 

Neither  Agree  nor  Disagree,  4 
Army  SOF  types. 

=  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  There  were  no 

Appendix  E,  Table  6.  There  are 

language  tasks  operators  need  to  perform  when  deployed  that  are  not  covered  by  the  OPI. 

Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree 
Dis  agree 

nor 

Agree 

Strongly  Agree 

Overall 

86 

3.71 

1% 

7% 

34%  H 

36% 

22% 

AFSOC 

1 

3.00 

0% 

0% 

100%  ■ 

0% 

0% 

WARCOM 

2 

3.50 

0% 

0% 

50%  [ 

50% 

0% 

MARSOC 

1 

3.00 

0%l 

0%l 

100%  E 

0%l 

0%I 

USASOC 

64 

3.77 

2%| 

5% 

33%  ■ 

38% 

23%  l 

CA 

6 

4.00 

0% 

0% 

33% 

33% 

33% 

MISG 

19 

3.79 

0% 

0% 

42%  I 

37% 

21% 

SF 

29 

3.83 

0%l 

7% 

28% 

41% 

24% 

Note.  1  =  Strongly  Disagree,  2  = 

=  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  : 

=  Agree,  5  =  Strongly  Agree.  Not  all  organizations  (e.g.,  JSOC,  TSOC)  are  represented  in  the  table.  There  were  no 

significant  differences  between  components  or  Army  SOF  types. 
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APPENDIX  F:  MI  LINGUISTS  AND  09L  SURVEY  RESPONSES 

Appendix  F,  Table  1.  How  related  is  the  OPI  to  what  operators  in  your  unit  do  on  the  job? 


Group  n 

M 

Not  Related 

Slightly  Related 

M  o  de  rate  ly  Re  late  d 

Related 

Very  Related 

MI  Linguists  or  09L  37 

2.81 

19%  1 

24%  I 

24% 

22%  I 

11% 

Note.  1  =  Not  Related,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related. 


Appendix  F,  Table  2.  The  content  of  the  OPI  is  clearly  related  to  what  operators  do  during  deployment. 


Neither  Agree  nor 

Group  n  M  Strongly  Disagree  Disagree  Disagree  Agree  Strongly  Agree 

MI  Linguists  or  09L  36  3.03  14%  14%  I  36%  B  28%  1  8%  I 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 

Appendix  F,  Table  3.  An  operator's  OPI  rating  accurately  reflects  their  ability  to  use  language  while  on  the  job. 


Neither  Agree  nor 

Group  n  M  Strongly  Disagree  Disagree  Disagree  Agree  Strongly  Agree 

MI  Linguists  or  09L  36  3.14  11%  14%  B  36%  B  28%  I,  11%  I 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 

Appendix  F,  Table  4.  Operators  who  perform  well  on  the  OPI  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  perform 
poorly  on  the  OPI. 


Neither  Agree  nor 

Group  n  M  Strongly  Disagree  Disagree  Disagree  Agree  Strongly  Agree 

MI  Linguists  or  09L  36  3.33  8%  8°/<J  39%  M  31%i  14%  I 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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Appendix  F,  Table  5.  An  operator's  OPI  rating  is  an  accurate  reflection  of  their  ability  to  perform  job/mission-related  tasks  in  the  target  language. 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  36 

3.03 

11%I 

19% 

36%  1 

22%  1 

11%I 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  F,  Table  6.  There  are  language  tasks  operators  need  to  perform  when  deployed  that  are  not  covered  by  the  OPI. 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  36 

3.50 

6%H 

6%l 

44% 

22%  1 

22%  1 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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APPENDIX  G:  CURRENTLY  IN  THE  TRAINING  PIPELINE  RESPONSES 


Appendix  G,  Table  1.  How  related  is  the  OPI  to  what  operators  in  your  unit  do  on  the  job? 


Group 

n 

M 

Not  Related 

Slightly  Related 

Moderately  Related 

Related 

Very  Related 

Pipeline 

8 

3.25 

13%  I 

25%  1 

13%  1 

25%  I 

25% 

Note.  1  =  Not  Related,  2  =  Slightly  Related ,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related. 


Appendix  G,  Table  2.  The  content  of  the  OPI  is  clearly  related  to  what  operators  do  during  deployment. 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

i 

3.43 

0%l 

14%  l 

43%  [ 

29%  [ 

14% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  3.  An  operator's  OPI  rating  accurately  reflects  their  ability  to  use  language  while  on  the  job. 


Group 

It 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

1 

2.86 

14%  I 

29% ! 

14%  [ 

43%  I 

0%l 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  4.  Operators  who  perform  well  on  the  OPI  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  perform 
poorly  on  the  OPI. 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

i 

3.86 

0%l 

0%l 

43%  [ 

29%  1 

29% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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Appendix  G,  Table  5.  An  operator's  OPI  rating  is  an  accurate  reflection  of  their  ability  to  perform  job/mission-related  tasks  in  the  target  language. 


Group 

11 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

1 

3.00 

14%  1 

14%  1 

29%. 

43%  1 

0%l 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree ,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  6.  There  are  language  tasks  operators  need  to  perform  when  deployed  that  are  not  covered  by  the  OPI. 


Group 

11 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

1 

4.00 

0%l 

0%l 

29% 

43%  1 

29% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  7.  Leaders  -  The  test  of  record  for  SOCOM  has  changed  from  the  DLPT  which  assesses  listening  and  reading  to  a  two-skill 
OPI,  which  assesses  listening  and  speaking.  What  is  your  reaction  to  this  change? 


Group 

n  M 

Strongly  Oppose 

Somewhat  Oppose 

Neither  Oppose  nor 
Support 

Somewhat  Support 

Strongly  Support 

Pipeline 

86  3.86 

l%l 

3%  I 

38%  l 

22%  1 

35%  l 

Note.  1  =  Strongly  Oppose,  2  =  Somewhat  Oppose,  3  =  Neither  Oppose  nor  Support,  4  =  Somewhat  Support,  5  =  Strongly  Support. 
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APPENDIX  H:  SOF  OPERATOR  OPI  PERCEPTIONS  BY  STANDARD  CHANGE  SUPPORT 


Appendix  H,  Figure  1.  How  related  is  the  OPI  to  what  you  do  on  the  job? 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


Appendix  H,  Figure  2.  The  content  of  the  OPI  is  clearly  related  to  what  I  do  during  deployment. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 
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Appendix  H,  Figure  3.  My  OPI  rating  accurately  reflects  my  ability  to  use  language  while  on  the  job. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


Appendix  H,  Figure  4.  Operators  who  perform  well  on  the  OPI  are  more  likely  to  successfully  use 
language  in  the  field  than  those  who  perform  poorly  on  the  OPI. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 
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Appendix  H,  Figure  5.  My  OPI  rating  is  an  accurate  reflection  of  my  ability  to  perform  job/mission- 
related  tasks  in  the  target  language. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


Appendix  H,  Figure  6.  There  are  language  tasks  I  need  to  perform  when  deployed  that  are  not  covered  by 
the  OPI. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 
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APPENDIX  I:  SOF  LEADER  OPI  PERCEPTIONS  BY  STANDARD  CHANGE  SUPPORT 


Appendix  I,  Figure  1.  How  related  is  the  OPI  to  what  operators  do  on  the  job? 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


Appendix  I,  Figure  2.  The  content  of  the  OPI  is  clearly  related  to  what  operators  do  during  deployment. 
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Appendix  I,  Figure  3.  An  operator’s  OPI  rating  accurately  reflects  their  ability  to  use  language  while  on 
the  job. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


Appendix  I,  Figure  4.  Operators  who  perform  well  on  the  OPI  are  more  likely  to  successfully  use 
language  in  the  field  than  those  who  perform  poorly  on  the  OPI. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 
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Appendix  I,  Figure  5.  An  operator’s  rating  is  an  accurate  reflection  of  their  ability  to  perform  job/mission- 
related  tasks  in  the  target  language. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


Appendix  I,  Figure  6.  There  are  language  tasks  operators  need  to  perform  when  deployed  that  are  not 
covered  by  the  OPI. 
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Note.  Opposed  =  opposed  the  standard  change,  Neutral  =  neither  opposed  nor  supported  the  standard  change,  Supported  =  supported  the  standard 
change. 


11/2/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011016] 


Page  49 


SOF  Language  and  Culture  Needs  Assessment  Project 


Oral  Proficiency  Interview  (OPI) 


APPENDIX  J:  COMMENT  CODE  DEFINITIONS 

SOF  operators  and  leaders  provided  comments  to  the  following  prompts: 

•  Provide  any  specific  feedback  that  you  have  related  to  the  standard  change. 

•  Responses  for  “Other”  problems  experienced  during  OPI  testing. 

•  Use  the  space  below  to  provide  any  specific  feedback  that  you  have  to  issues  you 
experienced  while  taking  the  OPI. 

•  Please  provide  any  specific  feedback  you  have  related  to  the  OPI. 

All  comments  were  content  analyzed  to  extract  common  themes.  The  resulting  themes  are  provided 
below  by  item  prompt  with  a  definition  and  verbatim  exemplar  comments  that  illustrate  the  theme.  For 
more  information  about  this  study’s  content  analysis  process,  please  refer  to  the  Methodology >  Report 
(Technical  Report  #  2010011002). 

Note:  Exemplar  comments  are  presented  verbatim  and  are  uncorrected  for  spelling  and  other  mistakes. 

Provide  any  specific  feedback  that  you  have  related  to  the  standard  change. 

•  Supporting  change  -  General 

o  Definition:  Respondents  made  a  comment  that  was  supportive  of  the  standard  change  but 
was  not  specific. 

■  “It  makes  more  sense.  We  interact  internationally  through  listening  and 
speaking.. .reading  is  often  just  a  bonus.” 

■  “I  like  it” 

•  Speaking  and  listening  are  the  most  important  proficiencies  for  operators 

o  Definition:  Respondent  believed  speaking  and  listening  were  most  important.  This  also 
includes  respondents  who  specifically  indicated  that  it  was  more  important  than 
listening/reading. 

■  “Reading  is  good  for  an  intelligence  analyst  but  for  an  operator  90%  of  his  time 
is  verbal  communication  with  a  foreign  national” 

■  “This  is  a  good  change.  Speaking  and  listening  are  far  more  important  than 
reading.” 

•  Need  listening,  speaking,  and  reading 

o  Definition:  Respondents  indicated  that  the  standard  should  reflect  speaking,  listening  and 
reading. 

■  “37F  operators  need  to  have  reading  skills  in  addition  to  listening/speaking.” 

■  “I  do  believe  that  there  is  a  need  for  individuals  to  learn  how  to  read  and  write.  It 
helps  the  individual  understand  the  grammer  rules  of  a  language  in  order  to 
support  continued,  more  advanced,  training.” 
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•  Not  familiar  with  the  change 

o  Definition:  Respondents  indicated  that  they  were  not  familiar  with  the  standard  change  or 
had  not  heard  about  it. 

■  “This  is  news  to  me;  I  wonder  how  long  it  will  take  to  implement  this  new 
change  throughout  SOF.” 

■  “I  did  not  know  this  had  changed.” 

•  None/NA/off  topic 

o  Definition:  Respondents  indicated  that  they  had  no  comment  concerning  the  standard 
change  or  made  a  comment  was  unrelated. 

■  “none  standards  change” 

■  “None” 

•  Opposing  change  -  General 

o  Definition:  Respondent  made  a  comment  that  opposed  the  standard  change  but  was  not 
specific. 

■  “OPIs  are  more  subjective  and  do  not  necessarily  indicate  the  students  true 
understanding  of  the  language.” 

■  “This  is  another  requirement  that  does  not  directly  support  a  teams  ability  to 
conduct  the  SOF  core  tasks.” 

•  Not  familiar  with  the  OPI 

o  Definition:  Respondents  reported  that  they  were  not  familiar  with  the  OPI. 

■  “I  have  never  taken  the  Two  Skill  OPI  so  I  have  no  idea  as  to  how  this  compares 
with  the  DLPT” 

■  “I  have  not  taken  the  new  test  so  I  cannot  comment.” 

•  Not  getting  paid  for  OPI  or  want  a  financial  incentive 

o  Definition:  Respondents  indicated  that  operators  should  gain  a  monetary  incentive  for 
doing  well  on  the  OPI. 

■  “Get  the  guys  paid  who  do  well  on  the  OPI.” 

■  “Do  I  get  paid  for  scoring  well  on  the  OPI?  Thats  what  I  care  about.” 

•  Skills  should  be  relevant  to  their  job 

o  Definition:  Respondents  commented  that  the  standard  should  reflect  skills  relevant  to 
their  job. 

■  “Doesnt  matter  what  test  battery  you  use,  as  long  as  its  relevant  to  our  job.” 

■  “The  test  should  include  those  subjects  that  the  soldier  will  use  down  range,  i.e. 
military  terms,  contract  negotiation,  conversation,  etc..” 

•  Need  to  match  training  to  new  testing  format 

o  Definition:  Respondents  indicated  that  their  training  was  not  adequate  for  the  testing 
requirement  or  that  training  should  match  the  OPI. 

■  “If  this  is  how  they  are  going  to  test  then  this  is  how  they  need  to  teach  the  info” 

■  “I  don  not  have  time  to  do  language  training  as  it  is.  if  we  change  the  test  i  will 
have  to  completly  retrain  my  self  in  my  language.” 
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•  Against  reading/DLPT 

o  Definition:  Respondents  made  a  comment  that  was  against  testing  on  reading  skills  or  the 
DLPT,  but  did  not  specifically  endorse  the  OPI. 

■  “A  reading  test  is  absolutely  non-essential,  particularly  in  Pashtu  or  Farsi.” 

■  “The  soldiers  ability  to  understand  the  written  portions  of  the  language  are  a 
critical  component  to  his  ability  to  communicate  in  a  language.  Does  he  need  to 
read  a  full  page  test  essay  and  then  pick  apart  the  subtle  meaning  -  probably  not. 
Does  he  need  to  be  able  to  read  street  signs,  graffiti,  storefront  signs,  addresses  - 
in  Cyrillic/  Persian/  Arabic/  Thai  characters  -  absolutely.  That  is  another  area 
where  the  DLPT  is  a  failure.  It  doesn't  test  the  soldier's  understand  of  what  is 
critical  for  him  to  know.  Should  speaking  be  a  part  of  the  test?  Yes.  But  clear 
guidelines  must  be  given  to  the  evaluator  regarding  the  left  and  right  limits.  I 
might  not  be  able  to  tell  you  a  verbatim  transaltion  of  something,  but  if  i  can  talk 
around  it  with  words  that  I  know  and  get  my  point  across,  mission  accomplished. 
If  I  can  ask  for  clarity  and  then  have  something  explained  to  me  in  a  way  I 
comprehend  -  mission  accomplished.” 

Responses  for  “Other”  problems  experienced  during  OPI  testing.15 16 

•  Technical  difficulties 

o  Definition:  Respondent  indicates  that  they  had  technical  difficulties  (e.g.,  issues  with 
phone  or  hearing  tester)  during  the  OPI. 

■  “could  not  hear  the  person  on  the  telephone  due  to  audio  difficulty” 

■  “difficulty  hearing  on  the  phone” 

•  Tester  problems 

o  Definition:  Respondent  references  problems  with  the  tester  during  their  OPI,  including 
problems  with  the  tester’s  attitude  or  ability  to  hear  their  questions. 

■  “instructor  speaks  too  softly” 

■  “Instructors  not  knowing  their  role  or  what  to  do  during  the  interview.” 

•  Different  dialect  used  in  field  than  in  test 

o  Definition:  Respondent  indicated  that  the  dialect  used  during  the  interview  was  different 
than  the  dialect  used  during  a  mission. 

■  “Different  dialect” 

■  “dialect  issues” 

•  Problems  with  scoring 

o  Definition:  Respondents  indicated  that  they  did  not  agree  with  the  way  the  OPI  was 
scored. 

■  “Instructor  not  well  qualified  to  grade” 

■  “NOT  ACCURATE  GRADING  SCALE” 


15  Only  SOF  operators  who  wrote  in  “other”  problems  they  experienced  during  OPI  testing  received  this  item. 

16  Some  respondents  referred  to  the  OPI  tester  as  an  “instructor.”  These  respondents  are  likely  describing  problems 
that  occurred  during  an  in-class  oral  proficiency  exam  (OPE),  which  is  not  an  official  OPI.  Another  possibility  is 
that  they  are  describing  a  face-to-face  OPI  conducted  at  DLI  where  some  instructors  are  testers. 
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•  Structure  of  the  test 

o  Definition:  Respondents  reported  problems  with  the  way  the  OPI  was  conducted. 

■  “telephone  conversations  are  more  difficult  than  face  to  face” 

■  “subjective  test” 

•  Other 

o  Definition:  Respondents  indicated  other  problems  that  were  not  included  with  previous 
themes. 

■  “Access  to  testers” 

■  “Test  began  in  the  wrong  language” 

Use  the  space  below  to  provide  any  specific  feedback  that  you  have  to  issues  you  experienced  while 
taking  the  OPI.17 

•  Problems  with  scoring 

o  Definition:  Respondents  commented  on  the  OPI’s  scoring  system. 

■  “the  OPI  was  scewd  since  it  was  not  fairly  graded  througth  the  test  process 
certain  instructors  gave  higher  marks  while  other  gave  lower  marks  there  was  no 
set  standarf ,18 

■  “Instructor  didn't  seem  qualified  enough  to  grade.” 

•  Telephone/audio  problems 

o  Definition:  Respondents  commented  on  audio  quality  or  technical  problems  experienced 
during  a  telephonic  OPI. 

■  “The  phones  were  terrible.  You  could  barely  hear  the  tester;  they  didn't  have  a 
volume  switch.  The  tester  was  uncooperative  when  asked  to  speak  up  or  louder.” 

■  “The  telephone  connection  made  it  nearly  impossible  to  understand  the 
individual  on  the  other  end  of  the  line.  I  continually  had  to  ask  the  tester  to  repeat 
themselves  due  to  the  poor  connection  which  was  interpreted  as  me  not 
understanding  the  original  question  or  comment  made  by  the  instructor.” 

•  Should  be  done  face-to-face 

o  Definition:  Respondents  believed  that  the  OPI  should  be  administered  in  person  instead 
of  over  the  telephone. 

■  “telephone  conversation  was  not  a  good  idea  for  the  OPI.... it  should  have  been 
person  to  person  conversation” 

■  “[. .  .]We  need  to  copy  the  environment  we  will  be  operating  under-  which  is  face 
to  face” 

•  Process  problems 

o  Definition:  Respondents  commented  on  problems  with  the  OPI  process  (e.g.,  how  it  was 
administered,  encounters  during  testing) 

■  “should  be  done  by  computer” 

17  Only  SOF  operators  received  this  item. 

18  Some  respondents  referred  to  the  OPI  tester  as  an  “instructor.”  These  respondents  are  likely  describing  problems 
that  occurred  during  an  in-class  oral  proficiency  exam  (OPE),  which  is  not  an  official  OPI.  Another  possibility  is 
that  they  are  describing  a  face-to-face  OPI  conducted  at  DLI  where  some  instructors  are  testers. 
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■  “Most  military  personnel  will  answer  questions  in  a  short,  concise  manner.  The 
instructors  told  me  that  I  should  ramble  on  tangents  to  display  my  vocabulary 
ability.  Although,  it  is  possible  to  train  to  do  this,  it  presents  unique  challenges 
while  being  evaulated.” 

•  Speaking  and  listening  are  important/Should  be  part  of  the  DLPT 

o  Definition:  Respondents  believed  that  speaking  and  listening  are  important  proficiencies 
to  assess  or  commented  that  speaking  and  listening  should  be  incorporated  into  the  DLPT 
test  process. 

■  “It  needs  to  be  part  of  the  DLPT  test.  The  DLPT  needs  to  cover  all  aspects” 

■  “shut  be  part  of  every  DLPT  taken” 

•  NA/None/Not  Relevant 

o  Definition:  Respondent  made  a  comment  that  indicated  they  had  no  response  or  the 
response  was  irrelevant  to  the  question. 

■  “We  did  an  OPI  after  our  language  immersion.  There  was  no  test  prior  to  our 
immersion  so  no  way  to  see  if  our  language  immersion  aided  us  in  our  ability  to 
communicate  in  French.” 

■  “NA” 

Please  provide  any  specific  feedback  you  have  related  to  the  OPI.19 

•  None/NA/taken  too  long  ago 

o  Definition:  Respondents  indicated  that  they  did  not  have  feedback  for  the  OPI,  that  the 
test  was  taken  too  long  ago  to  comment,  or  their  response  did  not  apply  to  item. 

■  “None  at  this  time.” 

■  “I  frankly  do  not  remember  much  about  the  OPI.” 

•  Good  gauge  of  language  proficiency/ability  to  communicate 

o  Definition:  Respondents  believed  the  OPI  was  a  good  measure  of  speaking  proficiency 
and  an  individual’s  ability  to  communicate. 

■  “It  evaluates  the  Operators  ability  to  effectively  hold  a  conversation  in  his  target 
language.” 

■  “OPI  is  a  great  tool  to  test  a  person  command  of  a  language.” 

•  Should  cover  military  related  topics  or  be  related  to  the  mission 

o  Definition:  Respondents  felt  the  OPI  should  cover  more  topics  related  to  the  military  or 
mission-related  tasks. 

■  “OPIs  should  be  tailored  to  each  MOS  -  i.e.  /  special  forces  should  have  OPI 
questions  more  related  to  FID  and  training  locals  /  PSYOPers  should  have  OPI 
questions  more  related  to  asking  questions  about  social  atmospherics  /  CAers 
should  have  OPI  questions  more  related  to  asking  questions  about  standard  of 
life” 


19  Some  respondents  referred  to  the  OPI  tester  as  an  “instructor.”  These  respondents  are  likely  describing  problems 
that  occurred  during  an  in-class  oral  proficiency  exam  (OPE),  which  is  not  an  official  OPI.  Another  possibility  is 
that  they  are  describing  a  face-to-face  OPI  conducted  at  DLI  where  some  instructors  are  testers. 
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■  “No  military  scenarios  were  given  at  all  on  the  OPI.  I  was  asked  to  rent  a  room  in 
Paris  France  and  describe  the  room  and  ask  questions  about  its  content.” 

•  Emphasis  on  the  OPI 

o  Definition:  Respondents  indicated  that  more  emphasis  should  be  placed  on  the  OPI 
during  training. 

■  “More  stress  must  be  placed  upon  the  OPI  test.  However,  in  order  to  do  well  on 
an  OPI,  the  soldier  MUST  RECEIVE  a  commensurate  amount  of  "language 
immersion  training."” 

■  “More  emphasis  on  the  OPI.” 

•  OPI  wasn't  relevant  (wrong  modality,  etc) 

o  Definition:  Respondents  indicated  that  the  OPI  was  not  relevant  to  their  job.  This 
includes  the  test  being  the  wrong  modality  (e.g.,  speaking  instead  of  reading,  etc). 

■  “Needs  to  have  more  practical,  job-related  conversations  that  involve 
negotiations,  commands,  etc.” 

■  “The  Korean  language  OPI  was  not  related  to  a  North  Korean-US  military 
scenario.  I  cannot  speak  for  War  on  Terror  relevant  OPI’s.” 

•  Need  more  monetary  incentives  for  OPI 

o  Definition:  Respondents  indicated  that  there  should  be  a  monetary  incentive  for  doing 
well  on  the  OPI. 

■  “Since  the  OPI  doesn't  count  toward  getting  paid  very  few  take  it.” 

■  “need  to  look  at  paying  operators  for  their  proficiency  on  the  OPI/OPE.  This  is 
where  we  make  our  money  on  target.” 

•  Too  subjective 

o  Definition:  Respondents  believed  that  the  OPI  was  too  subjective  as  a  measure  of 
speaking  proficiency. 

■  “OPI  is  too  subjective.  I  took  the  OPI  and  spoke  for  45  minutes  on  the  phone  and 
received  a  0+/0+  rating.  AFTER  45  MINUTES  OF  SPEAKING.  But  scored  a 
2+12+  on  the  DLPT.” 

■  “The  OPI  is  subjective  to  the  instrucor  administering  it.  I  do  not  agree  that  it  is  an 
accurate  assessment  of  an  individuals  ability  to  communicate.  The  problem  is 
that  the  instructors  are  basing  the  test  on  a  number  of  things,  but  most  important 
to  them  is  grammar  and  pronounciation.  If  you  are  able  to  communicate  your 
thoughts  and  ideas  to  your  target  audience  I  do  not  think  it  is  overly  important 
that  the  individual  did  not  conjugat  the  verb  or  some  other  grammatical  error, 
because  the  person  you  are  speaking  to,  will  understand  what  you  are  trying  to 
say  and  often  times,  if  you  do  not  know  the  word  or  cannot  remember  it,  you  can 
simply  ask  them.  During  the  OPI,  all  of  this  is  counted  against  you.” 

•  Not  effective  for  reading  needs  or  not  good  replacement  for  DLPT 

o  Definition:  Respondents  believed  that  the  OPI  was  not  effective  for  reading  proficiency 
needs  or  that  it  was  not  a  good  replacement  for  the  DLPT. 

■  “As  a  PSYOP  NCO,  reading  is  a  critical  language  skill.  It  is  obviously  not 
covered  by  the  OPI.” 
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■  “As  stated  before,  I  am  a  proponent  for  the  DLPT  5  as  the  standard  of  measure 
with  proper  language  training  at  the  DLI/university  level,  so  that  operators  master 
the  language  concepts.  My  belief  is  that  the  OPI  is  a  quick  fix  to  get  a  SF 
candidate  through  the  SFQC,  but  he  will  not  retain  his  language  proficiency  once 
he  reaches  the  operational  Group.  There  is  also  no  incentive  tied  to  the  OPI,  i.e. 
no  extra  money.  An  individual  who  would  test  at  a  DLPT  2/2/2  and  receive  the 
$200/month  starting  at  the  school  house  would  be  more  inclined  to  maintain  his 
language  proficiency.” 

Needs  to  be  done  face-to-face 

o  Definition:  Respondents  indicated  that  the  OPI  should  be  done  face  to  face  versus  on  the 
telephone. 

■  “OPI  needs  to  be  done  face  to  face  and  not  over  the  phone.  OPI  should  be 
general  conversation,  if  you  can  understand  and  speak  the  language  you  will  do 
fine.” 

■  “OPI  testers  need  to  be  in-person  not  VTC.” 

Cannot  use  dialect 

o  Definition:  Respondents  indicated  that  they  could  not  use  a  particular  dialect  while 
testing  on  the  OPI. 

■  “School  taught  versus  native  dialects  vary  greatly,  especially  in  Africa.” 

■  “A  slang,  dialect,  or  military  terminologies  OPI  should  be  administered  in 
addition  to  the  MSA  OPI.  The  problem  encountered  when  only  being  trained  in 
MSA  is  that  the  Local  Nationals  can  understand  you,  but  if  not  training  in  slang, 
dialect  or  military  terminologies,  it  can  be  difficult  to  understand  the  Local 
Nationals.” 

Not  mandatory 

o  Definition:  Respondents  indicated  that  they  did  not  have  to  take  the  OPI. 

■  “A  waste  of  my  time.  Instructor  wasn't  qualified  enough,  and  I  didn't  even  have 
to  take  it  in  the  first  place.” 

■  “The  OPI  is  currently  not  mandatory,  which  is  a  problem.  This  should  be  a  part 
of  annual  testing  [. . .]” 

Problem  with  Navy  requiring  baseline  DLPT  levels  before  taking  OPI 

o  Definition:  Respondents  commented  on  the  current  Navy  requirement  of  scoring  at  a 
given  level  on  the  DLPT  before  taking  the  OPI. 

■  “Current  Navy  requirements  are  that  in  order  to  take  the  OPI  a  SOF  operator 
must  score  baseline  payable  proficiency  on  the  DLPT.  Conversational  ability  is 
the  most  difficult  aspect  of  a  language  to  develop  and  retain,  as  well  as  the  most 
useful  in  the  operational  environ.  Current  testing  protocols  greatly  undervalue 
verbal  communication,  in  either  a  human  oriented  capacity  or  to  qualify  for 
incentives.” 
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